Hurence / logisland

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Home Page:https://logisland.github.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Implement a cache feature for EnrichRecordsElasticsearch processor

mathieu-rossignol opened this issue · comments

Suppose you receive multiple web events from same user and you use EnrichRecordsElasticsearch processor to enrich webevents (records) with additional user information from an ES index. You don't want to grab the same information again and again if the additional fields from the ES user index are not changing often. In that case, with a cache feature for the EnrichRecordsElasticsearch processor, you may improve a lot performances as you won't grab additional info from ES at each webevent.