Uncaught ECONNRESET exception

[Pete Edwards]

When I use Needle to check the following link the process fails with no error messages because of an uncaught exception. This makes it impossible to use in another system as it takes down the whole process.

I have found a solution but I do not understand the consequences of my change, nor do I know how to write tests for it. I hope someone else will be able to pick this up. Someone has obviously thought about this before and added some code but commented it out:

needle/lib/needle.js

Lines 722 to 727 in f9d554e

 // out.on('error', function(err) { 
 //   had_error(err); 
 //   if (err.code == 'ERR_STREAM_DESTROYED' || err.code == 'ERR_STREAM_PREMATURE_CLOSE') { 
 //     request.abort(); 
 //   } 
 // }) 

My solution was to add the following in the same place:

    out.on('error', function(err) {
      had_error(err)
    });

Then when I run the test I get output: Error: aborted

Test:
DEBUG=needle node bin/needle get https://webharvest.gov/peth04/20041014212036/http://la.water.usgs.gov/pdfs/QW_DomWells_Chicot.pdf

  needle Making request #1 {
  agent: null,
  localAddress: undefined,
  lookup: undefined,
  protocol: 'https:',
  host: 'webharvest.gov',
  port: 443,
  path: '/peth04/20041014212036/http://la.water.usgs.gov/pdfs/QW_DomWells_Chicot.pdf',
  method: 'get',
  headers: {
    accept: '*/*',
    'user-agent': 'Needle/3.2.0 (Node.js v20.4.0; linux x64)',
    'accept-encoding': 'gzip, deflate, br',
    host: 'webharvest.gov'
  }
} +0ms
  needle Got response 302 {
  location: '/peth04/20041016222033/http://la.water.usgs.gov/pdfs/QW_DomWells_Chicot.pdf',
  'content-length': '0',
  date: 'Wed, 12 Jul 2023 08:21:15 GMT',
  server: 'Unknown',
  'strict-transport-security': 'max-age=63072000; includeSubDomains; preload'
} +538ms
  needle Redirecting to https://webharvest.gov/peth04/20041016222033/http://la.water.usgs.gov/pdfs/QW_DomWells_Chicot.pdf +1ms
  needle Making request #2 {
  agent: null,
  localAddress: undefined,
  lookup: undefined,
  protocol: 'https:',
  host: 'webharvest.gov',
  port: 443,
  path: '/peth04/20041016222033/http://la.water.usgs.gov/pdfs/QW_DomWells_Chicot.pdf',
  method: 'GET',
  headers: {
    accept: '*/*',
    'user-agent': 'Needle/3.2.0 (Node.js v20.4.0; linux x64)',
    'accept-encoding': 'gzip, deflate, br',
    host: 'webharvest.gov'
  }
} +0ms
  needle Got response 200 {
  'x-archive-orig-server': 'Microsoft-IIS/5.0',
  'x-archive-orig-etag': '"0baf7b02818c41:14c2"',
  'x-archive-orig-connection': 'close',
  'x-archive-orig-content-type': 'application/pdf',
  'x-archive-orig-last-modified': 'Thu, 01 Apr 2004 20:34:12 GMT',
  'x-archive-orig-content-length': '15081274',
  'x-archive-orig-date': 'Sat, 16 Oct 2004 22:20:48 GMT',
  'x-archive-orig-accept-ranges': 'bytes',
  'content-type': 'application/pdf',
  'content-length': '15081274',
  date: 'Wed, 12 Jul 2023 08:21:15 GMT',
  server: 'Unknown',
  'strict-transport-security': 'max-age=63072000; includeSubDomains; preload'
} +741ms
  needle Error: aborted
  needle     at connResetException (node:internal/errors:720:14)
  needle     at TLSSocket.socketCloseListener (node:_http_client:455:19)
  needle     at TLSSocket.emit (node:events:524:35)
  needle     at node:net:334:12
  needle     at TCP.done (node:_tls_wrap:631:7) +21s
  needle Error: aborted
  needle     at connResetException (node:internal/errors:720:14)
  needle     at TLSSocket.socketCloseListener (node:_http_client:455:19)
  needle     at TLSSocket.emit (node:events:524:35)
  needle     at node:net:334:12
  needle     at TCP.done (node:_tls_wrap:631:7) +0ms