Giters

Hurence / logisland

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Home Page:https://logisland.github.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Improve performance of the IncrementalWebSession processor

MiniPlayer opened this issue · comments

commented

To be sure to be sync in events processing, the processor do some thread sleeping and refresh of ES indices.
It would be good to add two degraded mode in term of fiability to improve performance.
Default mode (Fast mode) => really fast (no refresh and no sleep)
Moderate mode => no sleep but refresh are done with a configurable time to wait for
Slow mode => sleep and refresh (with a configurable time to wait for refresh)

property name : processing.mode , values : [FAST, MODERATE, SLOW]
by default the value is fast
The configurable time to wait for refresh is configurable by

property name : es.refresh.wait.time.ms which should be a long (milliseconds), default to 100000