waltzofpearls / kitkat

Command line Kinesis Streams producer and consumer

kitkat

Kitkat is a command line Kinesis Streams producer and consumer.

In producer mode kitkat reads messages from stdin, delimited with new line (\n), and produces them to the provided Kinesis Streams (--stream or -s)

In consumer mode kitkat reads messages from Kinesis Streams shards and prints them to stdout.

You will be able to produce and consume messages in KPL aggregated record format. This feature is currently under development.

Getting started

To install kitkat locally, make sure you have Go 1.10+ on your computer, and then run

go get -u github.com/waltzofpearls/kitkat

To see the usage of kitkat, run

$> kitkat --help

# Producer usage
$> kitkat p --help
$> kitkat produce --help

# Consumer usage
$> kitkat c --help
$> kitkat consume --help

Examples

Read messages from stdin, produce to my-demo-stream

$> kitkat p -s my-demo-stream

# Use a non default AWS profile name
$> AWS_PROFILE=my-profile kitkat p -s my-demo-stream

# Enable verbose mode, which prints shard ID and sequence number
# from the newly produced message
$> AWS_PROFILE=my-profile kitkat p -s my-demo-stream -v

# Tail syslog and pipe to kitkat producer
$> tail /var/log/syslog | AWS_PROFILE=my-profile kitkat p -s my-demo-stream -v

Consume and tail messages from my-demo-stream

$> kitkat c -s my-demo-stream

# Use a non default AWS profile name
$> AWS_PROFILE=my-profile kitkat c -s my-demo-stream

# Enable verbose mode, which prints stream info, message shard ID
# and message sequence number
$> AWS_PROFILE=my-profile kitkat c -s my-demo-stream -v

Enjoy!