jackc / pgx

PostgreSQL driver and toolkit for Go

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

my pool keeps running out of connections

salmanb opened this issue · comments

I'm using pgxpool in a kubernetes based http that connects to a database in AWS RDS. The prod deployment of the app is running pgx/v4 and creates a new connection to the DB for each request which then go through pgbouncer and that has been working fairly smoothly for a few years. We do run into the occasional memory leak issue from connections not closing properly but a quick restart fixes that.

I'm trying once again to use pgxpool using v5 release, but I'm not certain what I'm doing incorrectly as the issue I keep running into is that eventually the pool just runs out of connections and the app just chokes. I've looked through my code to make sure I'm closing rows where needed, have tried to acquire and release connections directly from and back to the pool but still no luck. The pgxpool connections are not going through pgbouncer -- this connects directly to the DB.

I've also tried tweaking various settings such as healthcheck timers, idle connection lifetime settings, etc., but the end result is pretty much the same every time. After a while, the pool connections are exhausted and my pool stats look as follows -- the app does not recover w/out a full restart:

acquireCount:110079 
acquireDuration:7.877843226s 
acquiredConns:32 
canceledAcquireCount:265 
constructingConns:0 
emptyAcquireCount:1385 
idleConns:0 
maxConns:32 
maxIdleDestroyCount:0 
maxLifetimeDestroyCount:927 
newConnsCount:1398
 totalConns:32]

These are the connection values I'm currently using:

	config.MaxConnLifetime = 10 * time.Minute
	config.MaxConnLifetimeJitter = 1 * time.Minute
	config.HealthCheckPeriod = 1 * time.Minute
	config.MaxConns = 32
	config.MinConns = 4

I'm not sure what I'm doing incorrectly, or if there are any other settings I should tweak here so that the pool doesn't get exhausted.

You might try looking at the pg_stat_activity view when the pool is exhausted. That view should show the last query each connection executed. That may give you information on where the connection leak is occurring.