Spring Boot Change Data Capture Example with Debezium (PostgreSQL + Debezium + Kafka)

Debezium

Debezium is an open source distributed platform for change data capture. Start it up, point it at your databases, and your apps can start responding to all of the inserts, updates, and deletes that other apps commit to your databases. Debezium is durable and fast, so your apps can respond quickly and never miss an event, even when things go wrong.

Technologies

Spring Boot

Prerequisites

Run this project

Note

Modify the KAFKA_ADVERTISED_HOST_NAME in docker-compose.yml to match your docker host IP (Note: Do not use localhost or 127.0.0.1 as the host ip if you want to run multiple brokers.)

Set your local ip to env variable.

export KAFKA_ADVERTISED_HOST_NAME=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 | awk '{print $2}')

Check the env variable

echo $KAFKA_ADVERTISED_HOST_NAME

Steps

1 . Clone project on your machine

git clone git@github.com:BatuhanKucukali/spring-boot-cdc-example-with-debezium.git

2 . Change directory

cd spring-boot-cdc-example-with-debezium

3 . Run Dependencies

docker-compose up

4 . Run Projects

cd customer-api
./mvnw spring-boot:run

cd customer-projection-consumer
./mvnw spring-boot:run

Create Connector

curl --location --request POST 'http://localhost:8083/connectors' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "customer-connector",
    "config": {
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "tasks.max": "1",
        "database.hostname": "postgres",
        "database.port": "5432",
        "database.user": "postgres",
        "database.password": "postgres",
        "database.dbname": "user",
        "database.server.name": "master-db"
    }
}'

You can also check api-doc

Create User with API

curl --location --request POST 'http://localhost:7010' \
--header 'Content-Type: application/json' \
--data-raw '{
    "firstName": "John",
    "lastName": "Doe",
    "email": "john@doe.test"
}'

You can also check api-doc

Check Database

customer-api database (PostgreSQL)

id	first_name	last_name	email
1	John	Doe	john@doe.test

customer-projection-consumer database (MariaDB)

id	first_name	last_name	email
1	John	Doe	john@doe.test

Sources

Getting help

If you're having trouble getting this project running, feel free to open an issue or email me!

About

Spring Boot Change Data Capture Example with Debezium (PostgreSQL + Debezium + Kafka)

Languages

Language:Java 100.0%