nodefluent / kafka-streams

equivalent to kafka-streams :octopus: for nodejs :sparkles::turtle::rocket::sparkles:

Home Page:https://nodefluent.github.io/kafka-streams/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Writting stream to topic doesn't write

axel-sirota opened this issue · comments

Hi! Following the wordcount.js example, I mimicked it into the following. Everything works, even if I get the end of the stream and write to a file or log it to console it works as supposed to, but it doesn't publish to another topic. Does that have a bug? Any clues?

const stream = kafkaStreams.getKStream();

stream
  .from("RawTempReadings")
  .filter(kv => parseInt(kv.value, 10) > -50)
  .filter(kv => parseInt(kv.value, 10) < 130)
  .tap(kv => console.log(kv))
  .to("ValidatedTempReadings");

const inputStream = kafkaStreams.getKStream();
inputStream.to("RawTempReadings");

const produceInterval = setInterval(() => {
  inputStream.writeToStream(`${Math.floor(Math.random()* 300)-100}`);
}, 100);

Promise.all([
  stream.start(),
  inputStream.start(),
]).then(() => {
  console.log("started..");
  setTimeout(() => {
      clearInterval(produceInterval);
      console.log("stopped..");
  }, 5000);
});

Node version: v15.13.0
OS: Mac
package.json:

"dependencies": { "kafka-streams": "^5.0.0", "kafkajs": "^1.15.0", "kafkajs-avro": "^1.0.4", "kafkajs-stream": "^0.0.12", "minimist": "^1.2.5", "node-rdkafka": "^2.10.1" } }

the .to function defaults to "send" mode which takes either a Buffer or a String. You need to change the mode of .to function to "buffer", which can take a Buffer or an Object.
See this example