gurbaj5124871 / workflow-schedular-runtime

Service to schedule tasks/jobs and process them in background at scale

Workflow Schedular Runtime

A highly scalable service to schedule workflow jobs/tasks with timers with persistance. The system is build to be distributed to scale with resilience.

Technologies Used

1. Temporal - It is a scalable and reliable runtime for Reentrant Processes. It's battle-tested at scale and used in production by companies like uber, netflix, snap (snapchat) etc. More about temporal:

   • Docs
   • TL;DR Intro Video

2. PostgreSQL - Used for persistance of workflows alongside temporal.

3. Nest - NodeJS TypeScript Framework.

Description

The service is a monorepo divided into two systems:

1. server: The service is responsible for serving requests to schedule webhook triggers into temporal
2. worker: The service responsible for polling workflows/jobs/tasks from temporal and execute the webhook triggers

System Design Architecture Diagram:

Prerequisites

1. Make sure Docker is installed on the system
2. Copy the .env file from .env.example

$ cp .env.example .env

Running the app

In the main directory, run the following command:

$ docker compose up -d

Helper tools

1. Swagger Docs: For testing out REST endpoints, Hosted at /api-docs path, for localhost
2. Temporal UI: For deep visibility into scheduled tasks/jobs, for localhost

How it works

1. POST /timers

  sequenceDiagram
     client ->> server: HTTP request with webhook url and time to delay
     server ->> temporal client SDK: Request to create workflow to trigger workflow with deadline
     temporal client SDK ->> temporal: gRPC call to register and persist the workflow task
     temporal ->> postgreSQL: persist the new workflow task
     postgreSQL ->> temporal: success response
     temporal worker SDK ->> temporal: poll temporal for new workflows to run
     temporal worker SDK ->> worker: run the workflow and start the timer
     worker ->> webhook trigger: trigger the post request on the provided url after deadline

Loading

2. GET /timers/:id

   sequenceDiagram
   client ->> server / temporal client: HTTP request to fetch the time left for task by id
   server / temporal client ->> temporal: "getDeadline" Query to fetch the timer expiration
   temporal ->> worker / temporal worker: query forwarded from client to workflow
   worker / temporal worker ->> temporal: returns the deadline/timeout
   temporal ->> server / temporal client: forwards the deadline/timeout from worker
   server / temporal client ->> client: Returns the difference between current time and deadline, if positive then returns time diff in seconds otherwise returns 0

Loading

Test

# install packages
$ npm i

# e2e tests
$ npm run test:e2e

Tasks

• POST /timers api to schedule the webhook trigger with timer and return id of the scheduled task. Shoots post http call with empty body after timeout
• GET /timers/:id api to fetch time left in seconds until timer expires. Return 0 if already expired
• Persist timer and timers shouldn't reset if system is down(They are persisted inside postgreSQL via temporal)
• If all the workers are down, and when they are back, they pick up the tasks to be processed and if timers are expired already they shoot post request
• Docker compose file to run the whole system by single command
• API test cases (not all cases covered yet)