Postgres Logical Replication
Overview
This is a POC to test out Postgres streaming logical replication in Rust using rust-postgres.
It requires:
- a running postgres database server.
- wal2json plugin should be installed.
- ensure
wal_level
config is set to 'logical' (ALTER SYSTEM SET wal_level = logical;
)
This example shows how to:
- create a publication
- create a replication slot
- read protocol messages
- read json encoded WAL updates
Currently, the example only replicates insert
statements on the given table. To try it run the following command with your db config:
DB_CONF="user=postgres password=password host=localhost port=5432 dbname=postgres" TABLE_NAME=<your-table-name> cargo run
It will start replicating every insert transactions on the source database as they arrive by following the Postgres Replication protocol. As soon as receives a new transaction, it replays it in an embeded duckdb database.
This POC uses Materialize's fork of rust-postgres for logical replication protocol implementation.
Helpful links
- Logical decoding example
- Replication issue on
rust-postgres
repo - Materialize's fork of
rust-postgres
with the patches required to support logical decoding - Replication example
- Instructure's Change data capture