kofr

Kafka connect CLI inspired by kaf and kubectl. Kofr wraps all of kafka connect REST API operations.

Installation
Usage
Configuration
Contributions

Installation

Archives of precompiled binary for Linux and Macos are available.
Using cargo. Requires rust to be installed on the system.
```
$ cargo install kofr
```

Usage

Add a new connect cluster

$ kofr config add-cluster dev --hosts http://localhost:8083

Change context to a cluster:

$ kofr config use-cluster dev

List available clusters

$ kofr config get-clusters

Connectors operations

List running connectors

$ kofr ls
 NAME                STATE     TASKS   TYPE     WORKER_ID
 load-kafka-config   RUNNING   1       SOURCE   127.0.1.1:8083
 test-connector      RUNNING   1       SINK     127.0.1.1:8083

List current connect cluster status

$ kofr cluster status
 Current Cluster: dev
 id : "GnW0xXSmqeO-t6CQVTJg"
 ...........................................
 HOST                      STATE
 http://localhost:8083     Online
 http://localhost:8080     Offline

Describing a connector

$ kofr cn describe <connector-name>
{
  "name": "test-connector",
  "config": {
    "name": "test-connector",
    "tasks.max": "1",
    "topics": "test-topic",
    "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector"
  },
  "connector": {
    "state": "RUNNING",
    "worker_id": "127.0.1.1:8083"
  },
  "tasks": [
    {
      "id": 0,
      "state": "RUNNING",
      "worker_id": "127.0.1.1:8083"
    }
  ],
  "type": "sink"
}

Create a connector from a configuration file, like kubectl.

$ echo '{"name":"loadd-kafka-config", "config":{"connector.class":"FileStreamSource","file":"config/server.properties","topic":"kafka-config-topic"}}' \
| kofr cn create -f -

Edit a running connector config, this will open $EDITOR, similar to kubectl.

$ kofr cn edit <connector-name>

Restarting, pausing and resuming a connector.

$ kofr cn restart <connector-name>
# alternatively, you can specify tasks options
$ kofr cn restart <connector-name> --include-tasks
$ kofr cn restart <connector-name> --only-failed

$ kofr cn pause <connector-name>
$ kofr cn resume <connector-name>

Patch a running connector with new configuration

$ kofr cn patch test-connector -d '{"file": "config/server.properties","name": "load-kafka-config","connector.class": "FileStreamSource","topic": "kafka-config-topic"'

Delete a running connector

$ kofr cn delete <connector-name>

Tasks operations

List tasks of a running connector

$ kofr tasks ls test-connector
Active tasks of connector: 'test-connector'
 ID   STATE     WORKER_ID        TRACE
 0    RUNNING   127.0.1.1:8083   -
 1    PAUSED    127.0.1.1:8083   -

Restarting a task

$ kofr task restart sink-connector 0

Getting a task status

$ kofr task status test-connector 0
{
  "config": {
    "task.class": "org.apache.kafka.connect.file.FileStreamSinkTask",
    "topics": "test-topic"
  },
  "status": {
    "id": 0,
    "state": "RUNNING",
    "worker_id": "127.0.1.1:8083"
  }
}

Connect plugins

List installed plugins on the cluster

$ kofr plugin ls

validate a given connector confiugration with a connector plugin.

$ echo '{"connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
        "tasks.max": "1",
        "topics": "test-topic"
}' | kofr plugin validate-config -f -

Configuration

By default, kofr reads config from ~/.kofr/config See examples for a basic config file.

Contributions

I welcome fixes for bugs or better ways of doing things or more importantly, code reviews. Kofr was made by the motivation of solving a problem when having to deal with multiple kafka connect clusters at my work was mundane and more importantly, learning rust wink-wink. I use it personally like I use kubectl or kaf. See issues for things I'd like to improve. If you have a question about the codebase or simply want to discuss something feel free to open an issue. See also kcmockserver which I used in testing kofr. It's incomplete and could use some contributions.

Related Projects

kcmockserver

A-Fayez / kofr