terryfe / Prozess

kafka library for node.js

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Prozess

Build Status Coverage Status

Prozess is a Kafka library for node.js This fork is wrote for Kafka version 0.8, still working on progress.

v0.8's protocol changes a log, so this client's APIs has changed against old version forked from cainus's repo, see examples below:

Kafka is a persistent, efficient, distributed publish/subscribe messaging system.

There are two low-level clients: The Producer and the Consumer:

##Producer example:

var Producer = require('./lib/Producer');

var producer = new Producer('topic', {host : 'test.kafka.rome.cluster.sina.com.cn', 'requireAcks': 1});

producer.on('error', function(err,res){
  console.log("error: ", err);
  console.log("res: ", JSON.stringify(res));
});


producer.on('connect', function(){
  producer.send(['123','456','789']);
});

producer.on('sent', function(error, res){
  console.log(JSON.stringify(res));
});

producer.connect();

##Consumer example:

var Consumer = require('./lib/Consumer');


options = {host : 'test.kafka.rome.cluster.sina.com.cn', topic : 'topic', partition : 0, offset : 0};
var consumer = new Consumer(options);
consumer.connect(function(err){
  if (err){
    console.log("Error: " + err);
    throw "could not connect to Kafka";
  }
  console.log("connected!!");  
  console.log("consuming: " + consumer.topic);
  
  consumer.on('message', function(messages){
    messages.forEach(function(m){
      console.log(m.payload.toString());
    });
    setTimeout(consumer.consume.bind(consumer),2000);
  });
  consumer.on('error', function(error){
    console.log(error);
  });

  consumer.on('again', function(){
    setTimeout(consumer.consume.bind(consumer),2000);
  });
  consumer.consume();
    
});

A Consumer can be constructed with the following options (default values as shown below):

var options = {
  topic: 'test',
  partition: 0,
  host: 'localhost',
  port: 9092,
  offset: null, // Number, String or BigNum
};

##Installation:

 npm install prozess

##Checkout the code and run the tests:

 $ git clone https://github.com/cainus/Prozess.git
 $ cd Prozess ; make test-cov && open coverage.html

##Kafka Compatability matrix:

Kafka 0.8 from gitPartial Supported
Kafka 0.7.2 ReleaseSupported
Kafka 0.7.1 ReleaseSupported
Kafka 0.7.0 ReleaseSupported
kafka-0.6Consumer-only support.
kafka-0.05Not Supported

Versions taken from http://incubator.apache.org/kafka/downloads.html

About

kafka library for node.js

License:MIT License


Languages

Language:JavaScript 100.0%