Real-time sync changes from a Ditto Big Peer application to external database.
Kafka is one way to implement CDC (Change Data Capture). The document change stream is a coarse user-consumable Kafka queue that allows you to react to changes made as documents are inserted, updated, or deleted from Ditto's Big Peer. This keeps an external database in sync with Ditto, which can be useful for integration with third-party services and business intelligence tools.
This is sample code that shows you how to build a small server-side application that acts as a Kafka sink & source between the Ditto Big Peer and an external instance of MongoDB.
- An instance of MongoDB.
- Basic understanding of Node.js
- Local installation of Node 16 (*note: KafkaJS does not work with Node 18 today).
- A Ditto application syncing with the Big Peer on the Ditto Portal that is on a dedicated cluster.
- Follow the tutorial to enable CDC on your Ditto app
- See the nodejs-mongo directory for example code to connect the Node.js instance to the Ditto Big peer as a Kafka sink to MongoDB.
MIT