Question: Best practise for query last activity within an measurement/field/tag index

[Paul Werner]

Hey everyone,
I was wondering, how I could fetch the timestamp of the latest entity within a filtered set of data.
At small scale everything works fine, when using something like this:

from(bucket: "xxx")
  |> range(start: 0, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "status") // filter to the wanted measurement/field/tag index
  // + maybe grouping if needed to get one time for the group
  |> sort(columns: ["_time"], desc: false)
  |> last()

at larger scale this fails, because the whole bucket is loaded into the memory. From my pow: reducing the range is not an option, because the latest activity can also be like 3 years old.

How can such a use-case be handled?

[github-actions]

This issue has had no recent activity and will be closed soon.

[Paul Werner]

please reopen

[github-actions]

This issue has had no recent activity and will be closed soon.

[Paul Werner]

+1

[ganko-pi]

Try to remove sort because it is redundant when you call last in the next line. Her are some tips how to optimize Flux queries: https://docs.influxdata.com/influxdb/cloud/query-data/optimize-queries/. Use pushdown functions as much as you can and avoid loading data into memory.

[github-actions]

This issue has had no recent activity and will be closed soon.