Giters

Hurence / logisland

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Home Page:https://logisland.github.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

IncrementalWebSession processor should bulk webevents in a monthly index

jerome73 opened this issue · comments

commented

Expected behavior and actual behavior.

The IncrementalWebSession processor should putbulk webevents in the ES index suffixed by the month ('yyyy.MM') instead of the day date ('yyyy.MM.dd') of the h2ktimestamp.
Therefore it could target one or a set of months depending on the h2ktimestamp of the collection of records processed.

Problem:
Today the IncrementalWebSession processor putbulks webevents in an ES index suffixed by the day date 'yyyy.MM.dd'. Pb may be in case of a rewind, happening at the beginning of a month it is possible to regenerate indexes (using day date) over the last month period of time.

Steps to reproduce the problem.

Specifications like the version of the project, operating system, or hardware.