michael-erasmus / kiner

Python AWS Kinesis Producer with error handling and thread support.

Kiner

Python AWS Kinesis Producer.

Features

• Error handling and retrying with exponential backoff
• Automatic batching
• Threaded execution

Inspired by the AWS blog post Implementing Efficient and Reliable Producers with the Amazon Kinesis Producer Library.

Installation

You can use pip to install Kiner.

pip install kiner

Usage

To use Kiner, you'll need to have AWS authentication credentials configured as stated in the boto3 documentation

from kiner.producer import KinesisProducer

p = KinesisProducer('stream-name', batch_size=500, max_retries=5, threads=10)

for i in range(10000):
    p.put_record(i)

p.close()