Hurence / logisland

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Home Page:https://logisland.github.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support for elasticsearch write StructuredStream

mathieu-rossignol opened this issue · comments

When reading from kafka and pushing to ES, this would avoid having to uselessly write to a kafka topic just to have the offset saved into the checkpoint location.

Also we currently use a BulkAdd to push into ES and using the native ES structured stream sink may improve performances.

Documentation
Internals about implementation

Implemented with PR #593