Custom WriteFailed Function: Unknown Errors

[pbegg]

I am unable to determine what is causing the following 2 errors. This is happening when I am using a custom writefailed function and the client is not connected to the internet. I have a Min retry time set to 10000 and max retry time of 20000, flush buffer set 30000. It only started to occur after ~24hours of no internet connection

using @influxdata/influxdb-client@1.23.0
node v12.19.0

First error:

Mar 15 17:52:34 ERROR: Write to InfluxDB failed (attempt: 5). Error: Max retry time exceeded. at st.sendBatch (/home/umpadmin/.signalk/node_modules/@influxdata/influxdb-client/dist/index.js:1:13133) at Timeout._onTimeout (/home/umpadmin/.signalk/node_modules/@influxdata/influxdb-client/dist/index.js:1:10629) at listOnTimeout (internal/timers.js:554:17) at processTimers (internal/timers.js:497:7)

Second error:

Mar 15 17:51:45 (node:2183) UnhandledPromiseRejectionWarning: RequestTimedOutError: Request timed out at ClientRequest.<anonymous> (/home/umpadmin/.signalk/node_modules/@influxdata/influxdb-client/dist/index.js:16:4115) at ClientRequest.emit (events.js:314:20) at TLSSocket.emitRequestTimeout (_http_client.js:715:9) at Object.onceWrapper (events.js:420:28) at TLSSocket.emit (events.js:326:22) at TLSSocket.Socket._onTimeout (net.js:483:8) at listOnTimeout (internal/timers.js:554:17) at processTimers (internal/timers.js:497:7)

I understand that the issue is pointed out in the error message I am just unsure of how to address it.

[Pavel Závora]

Can you please include the exact version the of client and node.js that you use?

[Pavel Závora]

1. The default configuration OOTB stops retrying of failed writes after 180_000 milliseconds: https://github.com/influxdata/influxdb-client-js/blob/master/packages/core/src/options.ts#L132-L148
   This error indicates that the write retry operations (all HTTP calls that try to write initial failed lines to InfluxDB) plus delays between the retries ((retries-1) * (minRetryDelay + (maxRetryDelay-minRetryDelay)* random() + retryJitter * random()) takes longer than maxRetryTime milliseconds.
2. RequestTimedOutError is issued when an underlaying socket of HTTP/HTTPS write operation to InfluxDB server times outs because of inactivity. It can happen because of a lost internet connection. The timeout is by default 10_000 milliseconds.

[pbegg]

I have updated my original post to include client version and node version.

I have just noted my writeOptions.writeFailed function returns Promise.resolve() in an if statement so I am guessing this is why I am receiving error 1. If I want to resolve this promise always it should be outside the if statement:
I will test tomorrow

     writeOptions.writeFailed = function (error,lines,failedAttempts) {
      networkCheck = false

      if (failedAttempts==options.writeOptions.maxRetries+1) {

        const bufferArray = lines
        app.debug(`Write Failed sending ${bufferArray.length} Metrics to Buffer`)
        fs.writeFile(`${bufferFileDir}-${Date.now()}.json`, JSON.stringify(bufferArray), (err) => {
          if (err) {
            app.debug(err);

          }
          else {
            app.debug("Saved to Buffer")

          }
        });
      return Promise.resolve()

      }
    }

2: I am specifically making a plugin with these changes to handle long periods of no internet connection. My device runs on a 4G connection and drops out of internet range periodically-multiple days. I dont understand what this error is actually implying and the effect it will have on the plugin. It does not always occur when I have no internet connection - I only noticed it after ~ 24hours of no internet.

[Pavel Závora]

Regarding the 1st error, it happens because the time of retry attempts exceeded the configured maxRetryTime milliseconds. It is not caused by the writeFailed callback. BTW the writeFailed callback is not called (in v1.23.0) when the lines expire sooner before maxRetries attempts are exhaused, it will be called in the next release with #421. Pls increase maxRetryTime to 1 month (30*24*60*60*1000) to be sure that all retry attempts are executed and the condition failedAttempts==options.writeOptions.maxRetries+1 matches everytime the client stops retrying the lines.

The 2nd error is unlikely caused by retrying the write operation. It is more likely that the rejection of a promise that is returned by writeApi.flush() or writeApi.close() is not handled, so please do writeApi.flush().catch(e => {/* write operation failed, the error is already logged by WriteApi */}).

Regarding the persistent file store in your example, you can also save lines.join('\n') and then retry write with writeApi.writeRecord(file contents as string).

As I already expressed in #425, you can make it all simpler by relying upon your own retryMechanism with your persistent store (files) ... configure the writeApi with maxRetries=0, make sure that you write less than batchSize records before flushing manually, and react upon errors of the flush operation by persisting the data you can then retry later. You may alternatively move the error handling logic from the flush error handling to the writeFailed handler.