zalando / patroni

A template for PostgreSQL High Availability with Etcd, Consul, ZooKeeper, or Kubernetes

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

synchronous_standby_names removed by Patroni in PostgreSQL synchronous replication mode

heysky opened this issue · comments

What happened?

We recently upgraded patroni from 2.1.3 to 3.2.1. It looks the behaviour of PostgreSQL synchronous replication mode changed.

In 2.1.3, we usually build a 3-node cluster with asynchronous replication and setup patroni. And then we enable PostgreSQL synchronous replication by setting synchronous_standby_names and synchronous_commit with patronictl edit-config and it pushes the parameters to each node. It works quite well.

But after upgrading to 3.2.1, when we set synchronous_standby_names and synchronous_commit with patronictl edit-config, the parameter synchronous_standby_names would be removed by patroni in postgresql.conf of primary node although I still can see it with command patronictl show-config.

Would you please advise if it's expected behaviour or a bug? Is there any parameter to control it?

How can we reproduce it (as minimally and precisely as possible)?

Build a 3-node cluster with asynchronous replication and setup patroni. And then enable PostgreSQL synchronous replication by setting synchronous_standby_names and synchronous_commit with patronictl edit-config.

What did you expect to happen?

We expecte to set synchronous_standby_names with patronictl edit-config in PostgreSQL synchronous replication mode.

Patroni/PostgreSQL/DCS version

  • Patroni version: 3.2.1
  • PostgreSQL version: 14.10
  • DCS (and its version): ETCD 3.5.10

Patroni configuration file

scope: PG-cluster
name: PG1

# log
log:
  level: DEBUG
  traceback_level: ERROR
  dir: /app/patroni/logs
  file_num: 10
  # file_size: 26214400

restapi:
  listen: xxxx:8008
  connect_address: xxxx:8008

etcd3:
  hosts: xxxx:2379,xxxx:2379,xxxx:2379


bootstrap:
  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
  # and all other cluster members will use it as a `global configuration`
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"


postgresql:
  listen: "*:1921"
  connect_address: xxxx:1921
  database: postgres
  data_dir: xxxx/pgdata
  bin_dir: xxxx/bin
  pgpass: xxxx/.pgpass
  callbacks:
    on_start: /xxxx/patroni_callback.sh
    on_stop: /xxxx/patroni_callback.sh
    on_role_change: /xxxx/patroni_callback.sh

  authentication:
    replication:
      username: xxxx
      password: xxxx
    superuser:
      username: xxxx
      password: xxxx
    rewind:
      username: xxxx
      password: xxxx

watchdog:
   mode: automatic # Allowed values: off, automatic, required
   device: /dev/watchdog
   safety_margin: 5

tags:
   nofailover: false
   noloadbalance: false
   clonefrom: false
   nosync: false

patronictl show-config

loop_wait: 10
master_start_timeout: 300
maximum_lag_on_failover: 1048576
postgresql:
  parameters:
    hot_standby: 'on'
    max_replication_slots: 10
    max_wal_senders: 10
    synchronous_commit: remote_apply
    synchronous_standby_names: PG2
    wal_level: replica
    wal_log_hints: 'on'
  use_pg_rewind: true
  use_slots: true
retry_timeout: 10
ttl: 30

Patroni log files

2024-03-15 09:43:12,712 INFO: Changed synchronous_commit from on to remote_apply
2024-03-15 09:43:12,712 INFO: Changed synchronous_standby_names from  to PG2
2024-03-15 09:43:12,714 INFO: Reloading Postgres configuration.

PostgreSQL log files

2024-03-15 09:43:12.716 CST,,,76383,,65efbd81.12a5f,7,,2024-03-12 10:27:13 CST,,0,LOG,00000,"received SIGHUP, reloading configuration files",,,,,,,,,"","postmaster",,0
2024-03-15 09:43:12.717 CST,,,76383,,65efbd81.12a5f,8,,2024-03-12 10:27:13 CST,,0,LOG,00000,"parameter ""synchronous_commit"" changed to ""remote_apply""",,,,,,,,,"","postmaster",,0
2024-03-15 09:43:12.717 CST,,,76383,,65efbd81.12a5f,10,,2024-03-12 10:27:13 CST,,0,LOG,00000,"parameter ""synchronous_standby_names"" changed to ""PG2""",,,,,,,,,"","postmaster",,0
2024-03-15 09:43:17.084 CST,"replica","",76399,"10.16.6.239:40888",65efbd81.12a6f,1,"streaming 0/8017670",2024-03-12 10:27:13 CST,4/0,0,LOG,00000,"standby ""PG2"" is now a synchronous standby with
 priority 1",,,,,,"START_REPLICATION SLOT ""pg2"" 0/8000000 TIMELINE 1",,,"PG2","walsender",,0
2024-03-15 09:43:22.722 CST,,,76383,,65efbd81.12a5f,12,,2024-03-12 10:27:13 CST,,0,LOG,00000,"received SIGHUP, reloading configuration files",,,,,,,,,"","postmaster",,0
2024-03-15 09:43:22.723 CST,,,76383,,65efbd81.12a5f,13,,2024-03-12 10:27:13 CST,,0,LOG,00000,"parameter ""synchronous_standby_names"" removed from configuration file, reset to default",,,,,,,,,"","p
ostmaster",,0

Have you tried to use GitHub issue search?

  • Yes

Anything else we need to know?

No response

Hi @heysky,
you are not supposed to manage synchronous_standby_names, but let Patroni do it using synchronous_mode: on. Please check the documentation for more details.