influxdata / influxdb-client-js

InfluxDB 2.0 JavaScript client

Home Page:https://influxdata.github.io/influxdb-client-js/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question: How to retrieve the retry buffer size at any time

pbegg opened this issue · comments

commented

With long buffers (such as if the internet is disconnected for days) it is possible the retry buffer is full and data will be lost. Is it possible to call a function that will return the current buffer size?

The retry buffer maximum capacity is maxBufferLines, which is 32_000 in the default configuration. The internal implementation sweeps ~30% of the oldest items after it detects that it would overflow after adding new lines. This retry buffer implementation is however not designed to keep data for days, it is designed to overcome short outages that can happen in cloud deployments. The default configuration retries for at most 3 minutes and with at most 5 retry attempts that are scheduled using an exponential random retry strategy (better described in #343). You can get the actual size of the retry buffer in a writeFailed callback using this.retryBuffer.size, it is (and will remain) a private API.

I think that it would be better and simpler to solely rely upon your own implementation of a retry mechanism. In such a case, you have to configure the WriteAPI with writeFailed callback (as shown in #420) to be informed about the lines that failed and return Promise.resolve() after the lines are persisted to a store of your choice. Such configuration ensures that there is no internal retry mechanism in charge. You have to then retry the failed writes on your own, for example by a scheduled repetitive action over the persisted data.