brianc / node-pg-copy-streams

COPY FROM / COPY TO for node-postgres. Stream from one database to another, and stuff.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Idle connections with "COMMIT"

DnOberon opened this issue · comments

Attempting to use this package leaves me with my connection pool getting exhausted on multiple attempts to run the function. Looking in PGAdmin I see all the connections are still there, idle with COMMIT as their query. I've thrown a console.log in the end stream function and can verify it's firing - as well as calling trying to client.release manually.

Any idea why this isn't giving up connections and is hanging on COMMIT?
image

    public async CopyFromHypertable(source: DataSourceRecord, options?: copyTableOptions): Promise<Result<ReadStream>> {
        return new Promise((resolve) => {
            PostgresAdapter.Instance.Pool.connect((err: Error, client: PoolClient, done: any) => {
                if (err) {
                    return resolve(Result.Failure('unable to secure postgres client'));
                }

                const stream = client.query(copyTo(this.hypertableCopyToStatement(source, options)));
                stream.on('error', done);
                stream.on('end', () => {
                    client.release();
                });

                return resolve(Result.Success(stream));
            });
        });
    }
    ```
    
    Query looks like - `COPY (SELECT * FROM y_2 WHERE time BETWEEN etc. etc.`.
    
    What's interesting is that I take the read stream from that function and write it to a file and that seems to terminate ok, it's just the clients not releasing for whatever reason.

Hello, thanks for the report.

I am not sure what API you are using to get the client from the pool, but it seems weird to call "done" in the 'error' case and 'client.release()' in the 'end' case. Are you sure a call to "done" in both cases would not be sufficient (and maybe necessary) to release the client to the pool in both cases ?

@jeromew I was just testing out different methods of releasing the client.

Turns out it wasn't even this package causing the issue but pg large object storage. Sorry, closing now.