emaxerrno / awesome-kafka

awesome-kafka

Kafka

• doctorkafka
• topicctl - A tool for easy, declarative management of Kafka topics. Includes the ability to "apply" topic changes from YAML as well as a repl for interactive exploration of brokers, topics, consumer groups, messages, and more.
• sarama - Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later).
• kadeck - Apache Kafka Monitoring, Kafka UI and data platform for Desktop or Web
• conduktor
• mirus - Mirus is a cross data-center data replication tool for Apache Kafka
• kareldb - A Relational Database Backed by Apache Kafka.
• kcache - An In-Memory Cache Backed by Apache Kafka.
• kafkaOffsetMonitor - A little app to monitor the progress of kafka consumers and their lag wrt the queue.
• kafka-lag-based-assignor Kafka partition assignor that distributes lag evenly across a consumer group.
• kafka-pixy - Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control.
• Burrow - Kafka Consumer Lag Checking
• kafka-tools - A collection of tools for working with Apache Kafka.
• schema-registry-ui - Web tool for Avro Schema Registry
• kafkacat - Generic command line non-JVM Apache Kafka producer and consumer
• kafka-jackson - Kafka Serializer, Deserializer, and Serde for Jackson JSON
• kafka-lag-based-assignor - Kafka partition assignor that distributes lag evenly across a consumer group
• kafkahq - Kafka GUI for topics, topics data, consumers group, schema registry, connect and more
• secor - Secor is a service implementing Kafka log persistence
• reactive-kafka - Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
• kafka-monitor
• chaperone - A Kafka audit system
• uReplicator - Improvement of Apache Kafka Mirrormaker
• kafka-websocket
• kafka-unit
• cruise-control - Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a kafka cluster.
• jocko - Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
• hermes - Fast and reliable message broker built on top of Kafka.
• kt - Kafka command line tool
• kafka-eagle - Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information.
• dockerkafka - DoctorKafka is a service for Kafka cluster auto healing and workload balancing
• kasper - Kasper is a lightweight library for processing Kafka topics.
• Yelp kafka-utils
• kafka-spark-consumer
• kafka-streams-cep - Complex Event Processing on top of Kafka Streams
• spring-kafka
• ksql-machine-learning-udf
• kafkabeat - Elasticsearch Beats for kafka
• sangrenel
• Strimzi - Operator for deploying and running Apache Kafka on Kubernetes and OpenShift
• Kafka Manager - Web-based tool for managing a Kafka cluster
• Kafdrop - Web UI for browsing Kafka topics and consumer groups

Kafka UI

• kafka-topics-ui
• kafka-connect-ui - Web tool for Kafka Connect
• kafdrop
• kafka-webview
• kafka-view
• cruise-control-ui
• real-time-ui-with-kafka-streams
• tsujun - Yet another Web UI for KSQL.
• kowl

Kafka Streams

• schema-registry-ui
• kafka-streams-consumer-offsets-to-json - A Kafka Streams process to convert consumer_offsets to a JSON-readable topic.
• kafka-operator
• kafkabalancer
• mockedstreams
• winton-kafka-streams - A Python implementation of Apache Kafka Streams

Kafka Connect

• Apache Camel Kafka Connect - 340+ Apache Camel components as Kafka Connect connectors
• kafka-connect-client - A kafka-connect REST api client for java
• Confluent Connector Hub
• kafka-connect-protobuf-converter - Protobuf converter plugin for Kafka Connect
• Maxwell
• kafka-connect-transformers
• kafka-connect-mq-source
• kafka-connect-jenkins
• toketi-kafka-connect-iohub
• connectctl - Manage kafka connect connectors easily
• kafka-connect-transform-common - Common Transforms for Kafka Connect.
• aiven-kafka-connect-transforms - A collection of Single Message Transformations (SMTs) for Kafka Connect.
• kafka-connect-file-pulse - a polyvalent, scalable and reliable, Kafka Connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka.
• aiven-kafka-connect-transforms
• kafka-connect-http

REST Proxy

• kafka-rest - Confluent REST Proxy
• strimzi-http-bridge - Strimzi Kafka Bridge (AMQP & HTTP)

KSQL

Schema Registry

• apicurio-registry - Apicurio API/schema registry (includes UI)
• schema-registry - Confluent Schema registry for Kafka
• ksql-jdbc-driver

Other Awesome Kafka

• https://github.com/jsroyal/ApacheKafka
• https://github.com/dharmeshkakadia/awesome-kafka
• https://github.com/infoslack/awesome-kafka
• https://github.com/anascotti/awesome-kafka-playground

Kafkaesque

• Pulsar
• Flink
• Spark
• Heron
• Beam
• Arrow
• Samza
• Nats
• Chronicle-Queue
• CorfuDB
• GenStage
• winton-kafa-streams - A Python implementation of Apache Kafka Streams
• faust Python Stream Processing
• redpanda
• Meteor – Lightweight, broker-less alternative to Kafka for message streaming

Learning

• Passionate Developer: Kafka Streams DSL vs processor API
• Mastering Kafka Streams
• Spark Streaming + Kafka Integration Guide

Blogs

• CodingJunkie Random Thoughts on Coding by Bill Bejeck
• Understanding Kafka with Factorio

Books

• Kafka Streams in Action
• Kafka in Action
• Kafka the Definitive Guide