alexshin / pgpool-ii-balancing-example

That's just a couple of examples of how to balance SQL-queries to different Postgres nodes

Overview

This small project shows some specifics of practical development with balancing SQL-queries through Pgpool-II

Practical examples to look into are available in ./scripts/opt/tests/ directory.

A current setup of Pgpool-II is:

• 1 primary write replica
• 2 different read replicas
• pgpool node
• all non-write queries go to read-replicas only

Environments

• Pgpool listens on <docker-host>:5555
• Username: postgres
• Password: pass
• Database: target_db

Requirements

1. Python 3.9+
2. Docker
3. Docker-compose
4. psql cli must be installed and available via PATH

Docker

This environment runs on a single machine. So latency should be very short.

I use docker-compose v2 because v3 doesn't provide ways to limit resources. Discussion is available on this issue

How to run

1. Start all docker containers and wait for DBs being replicated:

docker-compose up

2. In other terminal set environment:

python -m pip install pipenv
pipenv install
pipenv shell
export PYTHONPATH=$(pwd):$(pwd)/scripts:${PYTHONPATH}

3. Load sample data:

./scripts/bin/load_data.py

# You can check all options with
./scripts/bin/load_data.py --help

4. Run tests

pytest ./scripts/opt/tests

Links

• Pagila - sample database for Postgres 12+
• https://www.refurbed.org/posts/load-balancing-sql-queries-using-pgpool/
• https://www.pgpool.net/docs/latest/en/html/runtime-config-load-balancing.html
• https://www.pgpool.net/docs/latest/en/html/example-kubernetes.html
• https://www.youtube.com/watch?v=oc_FQHq2dPI