Asynchronously sink Kafka topics into Pandas data frames
We wanted a tool aimed at feeding a Pandas dataframe starting from a Kafka consumer interactively. Right now, we have to this manually, nor there are real solutions for it. This is another example for the same use-case.
There is this tentative storing a single poll retrieved batch in a Pandas dataframe. We want to actual keep our dataframe up to date with the queried data, cross partition, and possibly cross topic.