sbcd90 / siddhi-kafka-cep

This is a simple CEP Engine leveraging the Kafka Streams platform

siddhi-kafka-cep

• This is a simple CEP Engine leveraging the Kafka Streams platform.

Key Features

• This project integrates Kafka Streams with WSO2 Siddhi CEP Engine and allows the users to leverage the best features of both platforms.

• The complex CEP patterns supported by Siddhi (link) can be used for processing events.

Getting started

• Compile the cep engine with maven.

mvn clean install

• Start the cep engine with the scripts in the scripts directory.

• Create a siddhi rule using the command

java -cp example-1.0-SNAPSHOT-jar-with-dependencies.jar org.apache.kafka.SiddhiRuleGenerator

• Start the event receiver using the command

java -cp example-1.0-SNAPSHOT-jar-with-dependencies.jar org.apache.kafka.SiddhiStreamsDataReceiver

• Publish data to the Engine using the command

java -cp example-1.0-SNAPSHOT-jar-with-dependencies.jar org.apache.kafka.SiddhiStreamDataGenerator

Architecture

Api

• The Java api documentation to create a new Siddhi Rule can be found here.

• The Java api documentation to publish & receive data from the CEP Engine can be found here & here.

Examples

• Examples for using the Java api s can be found the example directory.

Using Kafka Producers & Consumers

• The standard Kafka Producers & Consumers can also be used to publish & retrieve data out of this engine.

Rest Api

• A simple REST server is included with this project.

Getting started with the Rest server

• Run the maven jetty plugin to start the server.

mvn jetty:run

Payloads

• To create a Siddhi Rule:

  POST http://localhost:8080/cep/{streamId}/rule

  Body:

   {
   	"topic": "t1",
   	"bootstrapServers": "localhost:9092",
   	"definitions": ["define stream siddhiStream1 (symbol string, price double, volume long);"],
   	"rule": "@info(name = 'query1') from siddhiStream1[price < 20.0] select symbol, price, volume insert into outputStream"
   }
  

• To post streams of data:

  POST http://localhost:8085/cep/{streamId}/stream/data

  Body:

    {
    	"topic": "t1",
    	"bootstrapServers": "localhost:9092",
    	"data": [["Rectangle", 19.0, 19], ["Square", 21.0, 21]]
    }