🧸 kcctl – Your Cuddly CLI for Apache Kafka Connect

This project is a command-line client for Kafka Connect. Relying on the idioms and semantics of kubectl, it allows you to register and examine connectors, delete them, restart them, etc. You can see what kcctl is about in this short video:

Installation

You can obtain early access binaries of kcctl (x86) for Linux, macOS, and Windows from here. This is a rolling release, new binaries are published upon each commit pushed to the kcctl repository.

Note: on macOS, you need to remove the quarantine flag after downloading, as the distribution currently is not signed:

xattr -r -d com.apple.quarantine path /to/kcctl-1.0.0-SNAPSHOT-osx-x86_64/

We're planning to publish kcctl binaries via SDKMAN! soon, too.

It is recommended to install the bash/zsh completion script kcctl_completion:

. kcctl_completion

Quickstart

Before you can start using kcctl you need to create a configuration context. A configuration context is a set of configuration parameters, grouped by a name. To create a configuration context named local, with the Kafka Connect cluster URL set to http://localhost:8083, issue the following command

kcctl set-context local --cluster http://localhost:8083

❗ Note that certain commands will require additional parameters, like bootstrap-servers and offset-topic.

Type kcctl info to display some information about the Kafka connect cluster. The command will use the currently active context, local in this case, to resolve the cluster URL.

Usage

Display the help to learn about using kcctl:

kcctl help
Usage: kcctl [-hV] [COMMAND]
A command-line interface for Kafka Connect
  -h, --help      Show this help message and exit.
  -V, --version   Print version information and exit.
Commands:
  info      Displays information about the Kafka Connect cluster
  config    Sets or retrieves the configuration of this client
  get       Displays information about connector plug-ins, connectors, and
              loggers
  describe  Displays detailed information about the specified resource
  apply     Applies the given file for registering or updating a connector
  patch     Modifies the configuration of a connector or logger
  restart   Restarts a connector or task
  pause     Pauses a connector
  resume    Resumes a connector
  delete    Deletes the specified connector
  help      Displays help information about the specified command

Development

This project uses Quarkus, the Supersonic Subatomic Java Framework.

If you want to learn more about Quarkus, please visit its website: https://quarkus.io/ .

Running the application in dev mode

You can run your application in dev mode that enables live coding using:

./mvnw compile quarkus:dev

To seed the command line arguments, pass the -Dquarkus.args option:

./mvnw compile quarkus:dev -Dquarkus.args='patch get connectors'

In dev mode, remote debuggers can connect to the running application on port 5005. In order to wait for a debugger to connect, pass the -Dsuspend option.

Packaging and running the application

The application can be packaged using:

./mvnw package

It produces the quarkus-run.jar file in the target/quarkus-app/ directory. Be aware that it’s not an über-jar as the dependencies are copied into the target/quarkus-app/lib/ directory.

The application is now runnable using java -jar target/quarkus-app/quarkus-run.jar. You should define an alias kcctl:

alias kcctl="java -jar target/quarkus-app/quarkus-run.jar"

Creating a native executable

You can create a native executable using:

./mvnw package -Pnative

You can then execute your native executable with: ./target/code-with-quarkus-1.0.0-SNAPSHOT-runner

As above, either define an alias kcctl or rename the resulting executable accordingly.

Updating the Completion Script

Build the application in JVM mode. Then recreate the completion script:

java -cp "target/quarkus-app/app/*:target/quarkus-app/lib/main/*:target/quarkus-app/quarkus-run.jar" \
  picocli.AutoComplete -n kcctl --force dev.morling.kccli.command.KcCtlCommand

Edit the completion scrpt kcctl_completion, replace all the quotes around generated completion invocations with back ticks, making them actual invocations of kcctl::

--- local CONNECTOR_NAME_pos_param_args="kcctl connector-name-completions" # 0-0 values
+++ local CONNECTOR_NAME_pos_param_args=`kcctl connector-name-completions` # 0-0 values

Currently, three kinds of completions exist: connector-name-completions, task-name-completions, and logger-name-completions.

Related Quarkus Guides

Picocli (guide): Develop command line applications with Picocli
Quarkus native apps (guide): Develop native applications with Quarkus and GraalVM

License

This code base is available ander the Apache License, version 2.

hpgrahsl / kcctl