This project simulates a system for processing orders (i.e., a purchase) of items (E.g.: an order of a fridge and a camera) which is constituted of 4 microservices: order, stock, payment and notification service.
Note: as the main objective of this project is to demonstrate Kafka, majority of microservice patterns are ignored as well as some best practices for simplicity/readability’s sake (E.g.: Transactional outbox and coding to the interface).
- Clone this repo.
- Run the docker compose file inside “resources/docker files/Run project locally” directory (docker-compose up), if mongo-express fails to initialize, simply re-run it.
- Run all the microservices (any order of initialization is fine).
- Simply run the docker compose file inside “resources/docker files/Run project on docker” directory (docker-compose up), if mongo-express fails to initialize, simply re-run it.
- Send orders through order service’s endpoint /api/v1/order specifying the amount of orders to send and the amount of items inside each order through query param o (order) and i (item).
- E.g.: localhost:8080/api/v1/order?o=2&i=3 will send 2 orders, each order contains 3 items within itself.
- You can easily check the final result of each order through notification service’s console or check each topic through kafkdrop (localhost:9000/).
- You can change the expected order result (success/failure) and other configurations through shared.properties inside common module.
Once a new order is received, the order service does the initial processing and sends a new event to kafka:
- If running locally, configurations of the microservices can be changed through shared.application located at common\src\resources\ (explanaition of relevant configurations are included in this file).
- Kafka may be tricky to handle proficiently duplications/idempotency. In this project the approach of enabling idempotent producer was used. Consumer and its commit offset strategy should be considered also. E.g.: Idempotent Kafka Consumer
- This projects uses 2 DBs, Postgres for Order service and MongoDB for Stock service, this is only to showcase microservices and Kafka with different DBs.
- Common module should be replaced in a real scenario for a better approach as externalized configuration pattern.
- For simplicity sake, Kafka producer/consumer are using a shared entity located in the common module, in a real scenario Avro/schema registry (included in the docker-compose file) is advised.
- A caveat of using Kafka Streams and inner join to process the results of an order being processed by order services is that it is time windowed, if for any reason a service takes longer than the window time to answer an order, the orchestrator will never process the confirmation of the order. A few alternatives is to use individual listeners in the orchestrator or outer join and schedule a task to verify the order after join window has closed.
- If a rollback is needed, orchestrator (order service) will send an event (order) that is consumed by all services involved in processing an order rather than individual rollback events (E.g.: rollback event to payment service) as usually maximum 1 service will fail and because of this, all other services will have to rollback and therefore, consume an event.