How to consume messages from multiple topics
Torchu opened this issue · comments
Víctor Cabrita Gómez commented
I'm consuming messages from different topics and for each subgroup of topics I want to process the messages in a different way.
Example:
const consumer = kafka.consumer({ groupId: 'my-group' });
await consumer.connect();
await consumer.subscribe({ topics: ['topic-A', 'topic-B', 'topic-C'] });
await consumer.run({
eachMessage: async ({ topic, partition, message, heartbeat, pause }) => {
if (topic === 'topic-A') {
myFuncA(message);
} else {
myFuncB(message);
}
}
});
Is this the proper way to do it or should I create 2 different consuming groups: one for 'topic-A' and another one for 'topic-B' and 'topic-C' ?
Andrii Shumada commented
yes, this is proper way how to do this. If you create multiple consumer groups - it will create 3x connections and 3x more requests to Kafka.