copyFrom stream's end function does not behave as expected

[prabhash-c]

I have this pseudo-code that illustrates the issue I am facing

let readStream ( a read stream from somewhere )
var stream = client.query(copyFrom('COPY my_table FROM STDIN'))

readStream.on('data', (d) => stream.write(d))
readStream.on('finish', () =>  stream.end() )
stream.on('error', (e) => console.log(e))

When I am trying to close the stream on the 'finish' event of the readStream, it errors out with a message from pg module itself

Error: Connection terminated
    at Connection.<anonymous> (/Users/path/to/code/node_modules/pg/lib/client.js:132:36)`

and the data is not written on the PG table. Is this a flaw or am I missing something?

Thanks a ton for your help!

[jeromew]

Hello,
I am not sure I understand how your pseudo code works.

a readable stream does not emit a 'finish' event so afaik this cannot work - https://nodejs.org/api/stream.html#event-end

why don't you just pipe the streams ? readStream.pipe(stream)

[prabhash-c]

I need to do some checking and transformation, and the input stream has a method to send individual records, that I need to use.

How do I indicate to the stream that it can end and no more data is coming? stream.end() seems like what is needed?

[jeromew]

copyFrom STDIN is a standard Writable stream so yes stream.end() is a way to indicate to the stream that no more data is coming.

The error message you have Error: Connection terminated seems to mean that your pg connection was closed by something (?) It should not be closed before the 'finish' event is received on the copyFrom stream.

[prabhash-c]

Okay, so it seems stream.end() triggers stream.on('error', (e) => console.log(e)) which was breaking my promise chain resulting in connection being terminated.

If there any logical reason why stream.end() triggers stream.on.('error') for pg-copy-stream

[jeromew]

the only reason I could see is if you close the pg connection before finish triggers on copyFrom. If you do that, you close the connection while the COPY operation is not yet finished.

apart from that, if you need more help, please write a reproducible test highlighting your problem.

[prabhash-c]

That was correct, my connection was getting closed before the finish trigger. Thanks a lot!