gwenshap / kafka-examples

Snippets and small examples demonstrating kafka features and configs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Consumer is not receiving any messages

chinastockproducts opened this issue · comments

We are trying to use the same from code , but consumer topic is not receiving any messages on target side
export CLASSPATH=/Users/sxe518/Documents/test/MirrorMakerHandler/target/TopicSwitchingHandler-1.0-SNAPSHOT.jar
bin/kafka-mirror-maker.sh --consumer.config config/consumer.properties --producer.config config/producer.properties --whitelist test_.* —message.handler com.shapira.examples.TopicSwitchingHandler --message.handler.args ‘test,test123’
MirrorMakerHandler.zip

--message.handler.args ‘test,test123’

A topic name cannot contain commas. Here you have made the prefix contain them

Those are 2 different topic names test is from source and test123 is on consumer side.

I think you're misinterpreting the arguments then

public TopicSwitchingHandler(String topicPrefix) {
        this.topicPrefix = topicPrefix;
    }

Therefore, you're setting this.topicPrefix="test,test123"

ok so we need to use test.test123 right?

Depends on your goal.

If you want to rename multiple topics, look at https://github.com/opencore/mirrormaker_topic_rename

i followed opencore with the arguments--message.handler.args ‘test,test123’ . But i dont see topic test123 receiving messages from test. Is it any issue related to version of kafka..we are using kafka 1.1.0

If your source topic name is test, then your whitelist is only watching topics starting with test_, so the test topic itself is ignored...

This project is adding a prefix. The other project is completely renaming a topic. Using the format <src>,<dest>;<src2>,<dest2>

If you look at the example given there, you'd see their source topic does contain an underscore - test_source

--message.handler com.opencore.RenameTopicHandler --message.handler.args 'test_source,test_target'

Meanwhile, this project does not have an underscore in the topic name, and serves a different purpose

--message.handler com.shapira.examples.TopicSwitchingHandler --message.handler.args dc1 --whitelist mm1

It adds dc1 onto mm1

JavaSessionize.avro.LogLinec reference unresolved in the code, kindly look into it

@ashuit87 No it isn't. Use mvn compile to generate it