sabbyanandan / partitions

Stream Partitions Demo

LOCAL

Rabbit:

app register --name fruits --type source --uri maven://com.example:fruitsource:0.0.1-SNAPSHOT
app register --name appender --type processor --uri maven://com.example:appender:0.0.1-SNAPSHOT
app import --uri https://dataflow.spring.io/rabbitmq-maven-latest


stream create fooz --definition "fruits --server.port=9003 | appender | websocket"
stream deploy --name fooz --properties "app.fruits.producer.partitionKeyExpression=payload.id,deployer.appender.count=3"

Kafka:

app register --name fruits --type source --uri maven://com.example:fruitsource:0.0.1-SNAPSHOT
app register --name appender --type processor --uri maven://com.example:appender:0.0.1-SNAPSHOT
app import --uri https://dataflow.spring.io/kafka-maven-latest

Static Partition:
stream create fooz --definition "fruits --server.port=9003 | appender --spring.cloud.stream.kafka.bindings.input.consumer.autoRebalanceEnabled=false --spring.cloud.stream.kafka.binder.autoAddPartitions=true | websocket"

Dynamic Partition:
stream create fooz --definition "fruits --server.port=9003 | appender --spring.cloud.stream.kafka.binder.autoAddPartitions=true | websocket"
stream deploy --name foozs12 --properties "app.fruits.producer.partitionKeyExpression=payload.id,deployer.appender.count=3"

To scale appender:

http post http://localhost:9393/streams/deployments/scale/fooz/appender/instances/4

To override partitioning characteristics:

echo '{"releaseName":"fooz","packageIdentifier":{"packageName":"fooz"},"updateProperties":{"app.fruits.producer.partitionCount":4,"app.appender.spring.cloud.stream.instanceCount":4}}' | http http://localhost:9393/streams/deployments/update/fooz

CF

app register --name fruits --type source --uri https://github.com/sabbyanandan/partitions/raw/master/jars/fruitsource-0.0.1-SNAPSHOT.jar --force
app register --name appender --type processor --uri https://github.com/sabbyanandan/partitions/raw/master/jars/appender-0.0.1-SNAPSHOT.jar --force
app register --name websocket --type sink --uri maven://org.springframework.cloud.stream.app:websocket-sink-rabbit:2.1.1.RELEASE --force
stream create foo --definition "fruits | appender | websocket"
stream deploy --name foo --properties "app.fruits.producer.partitionKeyExpression=payload.id,deployer.appender.count=3"

K8S

app register --name fruits --type source --uri docker:sabby/fruitsource:0.0.2-SNAPSHOT --force
app register --name appender --type processor --uri docker:sabby/appender:0.0.3-SNAPSHOT --force
app register --name websocket --type sink --uri docker:springcloudstream/websocket-sink-rabbit:2.1.1.RELEASE --force

stream create foo --definition "fruits | appender | websocket"
stream deploy --name foo --properties "app.fruits.producer.partitionKeyExpression=payload.id,deployer.appender.count=3"