Kafka CLI
Apache Kafka is a high-throughput, distributed, publish-subscribe messaging system.
Requirements
How to build project ?
Go Lang must be installed in order to build the project.
You can used the command go get can be used to retrieve and build the project directly from GitHub.
go get github.com/fhussonnois/kafkacli/cmd/...The script ./release.sh can be used to package project tar.gz for all OS/architectures.
Setting defaults Host/Port for use by the CLI
Setting your host/port for use by the CLI tools can be done in a number of ways, here are some approaches:
-
Set the HOST and PORT environment variables.
To set these variables on Linux, macOS, or Unix, use export:
export SCHEMA_REGISTRY_HOST=your_schema_registry_host export SCHEMA_REGISTRY_PORT=your_schema_registry_portexport KAFKA_CONNECT_HOST=your_kafka_connect_host export KAFKA_CONNECT_PORT=your_kafka_connect_port -
Set the HOST and PORT in the CLI profile properties file on your local system, located at:
- ~/.kafkacli/host on Linux, macOS, or Unix
This file should contain lines in the following format:
# Kafka Connect kafka_connect_host = your_kafka_connect_host kafka_connect_port = your_kafka_connect_port # Schema Registry schema_registry_host = your_schema_registry_host schema_registry_port = your_schema_registry_port -
Set the HOST and PORT while specifying command to run :
The CLI commands accept both arguments -host and -port
Kafka Connect CLI
A simple Command line interface (CLI) to manage connectors through the Kafka Connect REST Interface.
Run command
./bin/kafka-connect-cliUsage
A simple Command line interface (CLI) to manage connectors through the Kafka Connect REST Interface.
Usage of ./bin/kafka-connect-cli: command [arguments]
The commands are :
list Listing active connectors on a worker.
config Getting connector configuration.
create Creating a new connector.
delete Deleting a connector.
delete-all Deleting all connectors.
pause Pausing a connector (useful if downtime is needed for the system the connector interacts with).
plugins Listing installed connectors plugins.
resume Restarting a connector.
restart-failed Restarting failed tasks for a connector.
status Getting connector status.
tasks Getting tasks for a connector.
scale Scaling up the number of tasks for a connector.
update Updating connector configuration.
version Getting a connect worker version.
Use "kafka-connect-cli help [command]" for more information about that command.
Examples
How to create a new connector instance ?
A new connector configuration can be passed through either json or properties file.
Usage of create: Creating a new connector.
The arguments are :
-config string
The connector configuration json string. (Required)
-config.json string
<file> The connector configuration json file. (Required)
-config.props string
<file> The connector configuration properties file. (Required)
-host string
The connector worker host address. (Required) (default "localhost")
-port int
The connector worker port. (Required) (default 8083)
-pretty
Pretty print json output.
How to display all connectors with failed tasks ?
Sometime it can be useful to quickly identify which connectors have failed tasks.
./kafka-connect-cli list -pretty -with-state failedConfluent Schema registry
A simple Command line interface (CLI) to manage Confluent Schema Registry.
Run command
./bin/schema-registry-cliUsage
A simple Command line interface (CLI) to manage Confluent Schema Registry.
Usage of ./bin/schemaregistrycli: command [arguments]
The commands are :
compatibility Getting subject compatibility level for a subject.
exist Checking if a schema has already been registered under the specified subject
get Getting a specific version of the schema registered under this subject
global-compatibility Getting the global compatibility level.
register Registering a new schema under the specified subject.
set-compatibility Setting a new compatibility level.
subjects Getting the list of registered subjects.
test Testing schemas for compatibility against specific versions of a subject’s schema.
versions Getting a list of versions registered under the specified subject.
Use "schema-registry-cli help [command]" for more information about that command.
Examples
How to register schema from URL ?
./bin/schema-registry-cli register -schema.url https://gist.githubusercontent.com/fhussonnois/e2d0d279dfa82ac038bd5776fe2486ae/raw/735ee897dac24e6c43cd1b5c4c84766701ed2c88/user.avsc \
-subject user -pretty
{
"id": 1
}How to retrieve only Avro schema from subject version ?
The option schema can be used to retieve only Avro schema of given subject without the version of the SchemaRegistry.
./bin/schema-registry-cli get -pretty -schema -subject user
{
"fields": [
{
"name": "name",
"type": "string"
},
{
"name": "favorite_number",
"type": [
"int",
"null"
]
},
{
"name": "favorite_color",
"type": [
"string",
"null"
]
}
],
"name": "User",
"namespace": "example.avro",
"type": "record"
}Contributions
Any contribution is welcome
Licence
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License