Kafka Connect Connector for S3
kafka-connect-storage-cloud is the repository for Confluent's Kafka Connectors designed to be used to copy data from Kafka into Amazon S3.
Kafka Connect Sink Connector for Amazon Simple Storage Service (S3)
Documentation for this connector can be found here.
Blogpost for this connector can be found here.
Development
To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the kafka-connect-storage-common FAQ for guidance on this process.
You can build kafka-connect-storage-cloud with Maven using the standard lifecycle phases.
Running Integration Tests
Integration tests are run as part of mvn install; however one needs to first configure the environment variableAWS_CREDENTIALS_PATH to point to a json file path with following structure:
{
"aws_access_key_id": "<key>",
"aws_secret_access_key": "<secret>"
}
Contribute
- Source Code: https://github.com/confluentinc/kafka-connect-storage-cloud
- Issue Tracker: https://github.com/confluentinc/kafka-connect-storage-cloud/issues
- Learn how to work with the connector's source code by reading our Development and Contribution guidelines.
License
This project is licensed under the Confluent Community License.
