masesgroup / KEFCore

Entity Framework Core provider for Apache Kafka

Home Page:https://kefcore.masesgroup.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add an option to use KNetCompactedReplicator

mariomastrodicasa opened this issue · comments

Is your feature request related to a problem? Please describe.
The tables are stored within topics using the key for Primary Key and value for all items, so it is possible to use KNetCompactedReplicator to manage the compacted topics.

Describe the solution you'd like
Add an options to choose between KNetCompactedReplicator and Apache Kafka Streams to retrieve the data from the topics.

Describe alternatives you've considered
N/A

Additional context
The types associated to KNetCompactedReplicator shall be managed like in #59

#59 will be applied later, the modification starts using KNetCompactedReplicator<string, string>

KNetCompactedReplicator shall be enhanced to cover some needs of KEFCore:

  • Add a flag, like IsStarted, indicating if the instance is started
  • Add a way to understand if the instance is in sync with the topic
  • Add a way to store keys, along with some metadata (offset, timestamp, etc), to avoid the local storage of values