Uncaught exception when Influx Cloud throws a 503 error

[Rick Bullotta]

Specifications

• Client Version: Latest NPM version 1.33.2
• InfluxDB Version: Influx Cloud, powered by TSM, storage engine V2, on AWS (free account)
• Platform: Node version 20.11.1 - issue occurs on both Windows and Linux clients

Code sample to reproduce problem

   var writeAPI;

    try {
        writeAPI = influxdb.getWriteApi(influxConfig.org,influxConfig.bucket)
        writeAPI.writePoint(point)
        writeAPI.flush()
    }
    catch(e) {
        console.error(e)
    }
    finally {
        try {
            if(writeAPI != undefined)
                writeAPI.close()
        }
        catch(eClose) {
            console.error(eClose)
        }
    }

Expected behavior

I would assume that if there was a write error, that it would be caught. It's actually surprising how often Influx cloud throws 503 (service unavailable) errors.

Actual behavior

Here is the error that is thrown, and the Node.js process exits when this error occurs.

ERROR: Write to InfluxDB failed. m [HttpError]: 503 Service Unavailable : upstream connect error or disconnect/reset before headers. reset reason: connection termination
at IncomingMessage. (/home/rick/uberbroker/node_modules/@influxdata/influxdb-client/dist/index.js:5:5671)
at IncomingMessage.emit (node:events:530:35)
at endReadableNT (node:internal/streams/readable:1696:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
statusCode: 503,
statusMessage: 'Service Unavailable',
body: 'upstream connect error or disconnect/reset before headers. reset reason: connection termination',
contentType: 'text/plain',
_retryAfter: 0
}
node:internal/process/promises:289
triggerUncaughtException(err, true /* fromPromise */);
^

m [HttpError]: 503 Service Unavailable : upstream connect error or disconnect/reset before headers. reset reason: connection termination
at IncomingMessage. (/home/rick/uberbroker/node_modules/@influxdata/influxdb-client/dist/index.js:5:5671)
at IncomingMessage.emit (node:events:530:35)
at endReadableNT (node:internal/streams/readable:1696:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
statusCode: 503,
statusMessage: 'Service Unavailable',
body: 'upstream connect error or disconnect/reset before headers. reset reason: connection termination',
contentType: 'text/plain',
_retryAfter: 0
}

Node.js v20.11.1

Additional info

This happens multiple times per day, and does not appear to be a catchable error. I'm curious:

1. why 503 errors happen so often on a hosted service
2. why this exception isn't caught - seems like a client bug

[Jakub Bednář]

Hi @RickBullotta,

Thank you for using our client and reaching out with your issue. The 503 error is considered retriable, and our client is designed to handle such scenarios with appropriate configuration.

To ensure the client retries upon receiving a 503 error, you can adjust the client options. These settings allow you to specify the behavior of the client in response to retriable errors, including the number of retry attempts and the interval between them.

For detailed information on how to configure these options and an example of advanced write operations, including error handling, please refer to our guide here: Write Advanced Example.

This example should provide you with a clear understanding of how to manage retries for 503 errors, ensuring your client can gracefully recover from temporary server unavailability.

Should you have any further questions or require additional assistance, please don't hesitate to ask.

Best Regards.

[Rick Bullotta]

Needed to add the await keyword to be able to catch the exception.