Leaking memory in postgres process

[rekado]

The following snippet causes the postgres process to eat all available memory.

require 'rdbi'
require 'rdbi-driver-postgresql'

dbh = RDBI.connect(:PostgreSQL,
                   host:     'localhost',
                   database: 'mydb',
                   username: 'user',
                   password: 'pass')

while true
  dbh.execute "SELECT false"
end

The memory consumption of the ruby process does not change much while this is running.

[rekado]

Possibly related: #33

[Mike Pomraning]

What version of rdbi? What version of the driver? (And can you prove it with a puts ? :)) Thanks.

[rekado]

rdbi reports its version as 1.0.0; I'm running this revision: 3742c7f (branch sth-leak)

rdbi-driver-postgresql reports its version as 0.9.2; I'm running this revision RDBI/rdbi-driver-postgresql@be571a9 (branch fix-execute-memory-leak)

[Mike Pomraning]

This code sample is, I think, dramatically leaking per-session prepared statement resources (server-side).

@Pistos, several related issues here:

1. RDBI::Driver::PostgreSQL::Statement does not deallocate any statement handles it creates
   
   Fix: @dbh.pg_conn.exec("DEALLOCATE #{@stmt_name}") rescue nil during @finish_block
2. pg::Statement cannot deallocate any statement handles it prepares
   
   Fix: Make statement handle names conform to identifier rules. There is no API call for prepared statement handle deallocation as there is for allocation, which is why we can prepare names (via API) which we cannot (via SQL) deallocate. Perhaps:
   
   @stmt_name = ('x' + Time.now.to_f.to_s).tr('.', '_')
3. The code sample allocates long-lived Statements
   If you don't explicitly finish a Result, nor implicitly do so via a block, then the Result and its Statement remain until gc'd.

1 and 2 can be fixed in the driver. 3 is the Way It Is. Compare memory if you dbh.execute("SELECT false").finish

[Pistos]

@pilcrow: Thanks for this info! I'll see what I can do to fix both the driver and our own code.

[Mike Pomraning]

Resolving as inapplicable; this is for the pg driver, not rdbi per se.

[Pistos]

Thanks, @pilcrow. I've applied your suggestions, and it does appear to repair the leak shown by this sample code. Hopefully we get good results in production as well.