reneradoi / kafka-connect-python

Python module for Kafka Connect REST API

kafka-connect-python

Python module for Kafka Connect REST API

Requirements

• Python (3.6)

Installation

There is no package provided (yet), add this module to the respective source code repository.

Examples

Create KafkaConnect REST Interface

from kafka_connect import KafkaConnect

connect = KafkaConnect(host='localhost', port=8083, scheme='http', user='my_user_name', password='my_password')

print(connect.api.version)

Create a connector using config dictionary

config = {
    "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
    "connection.url": "jdbc:postgresql://localhost/testdb?user=testuser&password=SecurePassword!",
    "key.converter": "io.confluent.connect.avro.AvroConverter",
    "key.converter.schema.registry.url": "http://localhost:8081",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "value.converter.schema.registry.url": "http://localhost:8081",
    "table.whitelist": "sampletable",
    "mode": "timestamp",
    "timestamp.column.name": "lastupdated",
    "topic.prefix": "test-0-"
}

connect.connectors['sample-connector'] = config

Update connector config

connector = connect.connectors['sample-connector']
connector.config['poll.time.ms'] = 500

List connector names

list(map(lambda c: c.name, connect.connectors))

Iterate over connectors and tasks

for connector in connect.connectors:
    for task in connector.tasks:
        task.restart()

Delete a connector

del connect.connectors['sample-connector']