PutInfluxDatabaseRecord_2 unable to write records with certain timestamps

[mpharding]

Hi all,

I have a PutInfluxDatabaseRecord_2 processor configured with a JSONTreeReader. The JSON records I'm trying to write are very simple e.g.

{"ts":"1667496077593", "temp":12.23, "uid":"1234"}
{"ts":"1667496295251", "temp":11.12, "uid":"1234"}

The examples above write to my influxdb just fine. However, if I try to write records that contain timestamps with several zeros at the end. The data never appears in database, enough though the flowfile transitions to the success path. For example, the following records will not write to the database.

{"ts":"1667498400000", "temp":14.3, "uid":"1234"}
{"ts":"1667509200000", "temp":14.3, "uid":"1234"}
{"ts":"1667520000000", "temp":14.3, "uid":"1234"}

This is my config:

This is my avro schema for the JSONTreeReader:

{
  "name": "ee",
  "type": "record",
  "namespace": "com.phyre.enviro.e.avro",
  "fields": [
	{
      "name": "uid",
      "type": "string"
    },
    {
      "name": "temp",
      "type": "float"
    },
    {
      "name": "ts",
      "type": "string"
    }  
  ]
}

I suspect this might have something to do with how Java handles scientific notation conversion. I've tried the ts as a long and a string but have the same problem either way.

I've also tried with a nanosecond timestamp and I still get the same issue.

I'm using version 1.17.0, as I'm unable to update our NiFi service at the moment but looking at the latest versions of these processors, I don't see any bug fix for this so would assume it applies to later versions.

Thanks,
Mike

[Jakub Bednář]

Hi @mpharding,

thanks for using our bundle.

{"ts":"1667498400000", "temp":14.3, "uid":"1234"}
{"ts":"1667509200000", "temp":14.3, "uid":"1234"}
{"ts":"1667520000000", "temp":14.3, "uid":"1234"}

These timestamps are from future. Do you use correct query in your DB? You have to use stop parameter with future date. For more info see -
https://docs.influxdata.com/flux/v0.x/stdlib/universe/range/#stop

Regards

[mpharding]

Ah! Thanks for that - wasn't aware I needed to include stop in the range() query.

Keep up the good work with the bundle!