dibbhatt / kafka-spark-consumer

High Performance Kafka Connector for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker.

dibbhatt/kafka-spark-consumer Issues

Can a higher version of Spark be supported? Spark 3.2.0 for example.
Closed 2 years ago1
Can a higher version of kafka be supported
Closed 2 years ago2
example doesn't build
Closed 3 years ago1
Trying to fetch Multi topic In Local , But It is showing warning like this
Closed 3 years ago2
Manipulation of offsetRanges in each batch
Updated 5 years ago
How to use the application to handle multi topic
Closed 5 years ago11
Offset is still updated when exception occurs during processing
Closed 5 years ago28
java.lang.NoClassDefFoundError: kafka/api/OffsetRequest
Closed 5 years ago22
Does this support spark structured streaming
Closed 5 years ago1
It works well in local model,but when I submit it in cluster model,the fixed rate is too small
Closed 6 years ago5
AbstractMethodError with Spark 1.6.0 and Kafka 0.10.2
Closed 7 years ago9
The Spark Streaming can not read kafka message
Closed 6 years ago8
May I have a Scala sample of messageHandler to filter out some playload which includes some strings?
Closed 6 years ago20
Why appear this exception information?
Closed 7 years ago8
How to use in kerberized context ?
Closed 7 years ago3
Consumer commit not proceeding the offset anymore
Closed 8 years ago16
java.lang.ClassNotFoundException: consumer.kafka.client.KafkaReceiver
Closed 8 years ago12
Not working with Spark 2.2.0
Closed 7 years ago11
How to recover the failed receiver on a partition which has exception of " Offsets out of range with no configured reset policy for partitions:"
Closed 6 years ago18
Hello, Compilation failed after changing Kafka version 0.10.0.0
Closed 6 years ago5
After long time running, the processing time of "ProcessedOffsetManager.persists(partitonOffset_stream, props)" incresing.
Closed 6 years ago13
Kafka Headers Support
Closed 6 years ago7
Exception: Could not compute split, block not found
Closed 7 years ago6
error: object ProcessedOffsetManager is not a member of package consumer.kafka
Closed 7 years ago1
The log of application contain too many the messages of LOG.info
Closed 7 years ago2
Consumer not able to read kafka topic and write to HDFS
Closed 7 years ago7
app client consumer doesn´t consume offsets from kafka topic
Closed 7 years ago43
ReceiverLauncher.launch can not get data
Closed 7 years ago5
java.lang.ClassNotFoundException: kafka.api.OffsetRequest
Closed 7 years ago2
Consumer how to use Kafka utilities like ConsumerOffsetChecker and get to show on Kafka Tool
Closed 7 years ago2
PartitionManager not writing Offset to Zookeeper if a record fetched from Kafka only contain one line.
Closed 7 years ago9
Contributing Receiver based Low Level Kafka Consumer from Spark-Packages to Apache Spark Project
Closed 8 years ago2
Released latest version 1.0.8 of the Receiver
Closed 8 years ago
mvn repository you provided is not working
Closed 8 years ago3
Python api?
Closed 8 years ago2
work fine few days, then can't commit offerset, no error log, and no info log about PartitionManager
Closed 8 years ago2
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/load/partitions
Closed 9 years ago10
Some jobs remain in processing state forever
Closed 8 years ago3
failure occurred when set spark.streaming.concurrentJobs from 1 to 2
Closed 8 years ago5
exception on simple example
Closed 8 years ago7
Static zookeeper hosts configuration
Closed 8 years ago2
How to decode byte messages from Kafka
Closed 9 years ago6
Running application on 3 node cluster, 3 instances of the same application are shown with different user
Closed 9 years ago5
Unable to receive data
Closed 9 years ago21
After some hours not processing because of executor fail error
Closed 9 years ago7
getting GC : outofmemory exception
Closed 9 years ago22
want to create more than 1 application but want single Kafka stream
Closed 9 years ago10
Kafka apache spark Streaming recive 0
Closed 9 years ago2
Not able to get streaming data
Closed 9 years ago4
MEMORY_ONLY, MEMORY_ONLY_* storage levels
Closed 9 years ago6