A bot that will watch for any updates on my university LMS. Also serves as a mirror for the LMS.
See the v1 tag for the old typescript + headless browser version
See the v2 tag fr the golang version without the headless browser
My uni use an LMS to give materials, submit assignments, etc. It's not perfect, but it works for the most part. Here's the dumb thing that I found, it has a notification feature but it never sends any notification whenever there is something new, like, why?? So, at one point, I did my assignment, but because there was no submission form until a few days later, I kinda forgot about it. Because there was no notification, I didn't know if the submission form was posted. Guess what, I didn't submit it because it was already overdue by the time I realised it.
It starts as a fun little project, but I ended up turning it into kinda serious project, which is why I even drew the architecture :p
Initially, it starts as a monolith service when I wrote it in Golang (The worker, API, and Frontend was served by Go hosted on fly.io), but then I decided to rewrite it to use Cloudflare Workers using Typescript.
Cloudflare Workers has its limitation, one of them is it can't run long running process. To tackle this, I split the presenter to be an external service so the worker can just call that. In the future, I might ended up with supporting multiple providers such as adding event to Google Calendar, sending the notification to Telegram, etc, but for now using Discord is enough.
You can visualise the system using this diagram that I made on Excalidraw:
The Notification Service which is used to send the notification to Discord is private, it requires an API key for authorisation. However, the REST API Presenter is open for public in case someone also wants to consume the data. It's available at https://siakad-worker.elianiva.my.id/api/subjects. It's containerised using Docker anad hosted at Fly.io because it's free. The reason why I chose to write it using Go is because it's very lightweight so it's suitable to host on a free platform such aasa Fly.io
I chose SvelteKit as the frontend because it's lightweight, fast to develop, has a good DX, and it's very easy to deploy on Cloudflare Pages. I also need it to do the server-side rendering because I don't want to do client-side fetching. It's accessible at https://cermin-siakad.elianiva.my.id. cermin meanas mirror in Bahas Indonesia. I made it because our LMS is not accessible at times, it's quite annoying to be honest.
Basically it watches the LMS periodically. By default, it will visit the LMS every 10 minutes and check if there is a new update or not. If there isn't, it will save the current snapshot of the courses in form of JSON collected by cheerio and compare it if there is an old version of that snapshot. If it sees any difference, it will send a notification to any webhook endpoint that is subscribed. At the moment there's only one webhook endpoint hosted on fly.io, which is used to send the notification to Discord.
You can see this flowchart to better visualise how the worker works:
After doing that process, the scraped data is stored into Cloudflare Workers KV. The REST API presenter will pick it up as its data source and serve it to public.
I don't feel like giving usage instruction for this app to random people on Github :p
Go figure it out yourself, or ask me in real life.
Idk, I picked it randomly, also, it sounds badass. It means 'notify' in welsh
- Main Worker
- Cloudflare Workers
- Cloudflare Workers KV - Stores the data and used as cache
- TailwindCSS - The styling library
- Typescript - Programming Language
- Hono - Used to manage routes easily for the presentation layer
- ofetch - A better fetch API
- Frontend Web Presenter
- Cloudflare Pages - Host the frontend
- Typescript - Programming Language
- SvelteKit - The frontend framework
- Cheerio - Used to scrape the data from html content
- Notification Service