org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/load/partitions

Question

org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/load/partitions

justinjoseph89 opened this issue 9 years ago · comments

Hi,
I am trying to run the consumer , but getting following error. Any idea why i am getting this?

INFO : org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
Exception in thread "main" java.lang.RuntimeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/load/partitions
at consumer.kafka.ReceiverLauncher.getNumPartitions(ReceiverLauncher.java:109)
at consumer.kafka.ReceiverLauncher.createStream(ReceiverLauncher.java:53)
at consumer.kafka.ReceiverLauncher.launch(ReceiverLauncher.java:36)
at consumer.kafka.client.Consumer.run(Consumer.java:72)
at consumer.kafka.client.Consumer.start(Consumer.java:43)
at consumer.kafka.client.Consumer.main(Consumer.java:95)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/load/partitions
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
at consumer.kafka.ReceiverLauncher.getNumPartitions(ReceiverLauncher.java:105)
... 5 more

Dibyendu Bhattacharya · Answer 1 · Tue May 05 2015 15:09:56 GMT+0800 (China Standard Time)

The Consumer example which is there at Github is a sample example . You need to modify the necessary properties to point to your Kafka cluster , your topic etc . The properties which already there are just an example .

Darpan Shah · Answer 2 · Wed Aug 24 2016 22:21:45 GMT+0800 (China Standard Time)

@justinjoseph89 Any idea how did you resolve this? I am having the same problem.
In fact, it was working everything but when i restart the whole system, and try to execute my program, its giving me this error.
@dibbhatt I did make all necessary changes. Initially, I created my topic with --partitions 6 --replication-factor 2. I got the same error. Is it related to partitions? I have a very similar another topic as well. However, its working fine. There is no difference between them except for the name.

Dibyendu Bhattacharya · Answer 3 · Wed Aug 24 2016 22:36:03 GMT+0800 (China Standard Time)

Can you check if the Kafka Partitions details is available in ZK ? This error mean in ZK , the Kafka Partition details is not there. Have you re-installed ZK ?

You can go to ZK command line by

/bin/zkCli.sh -server yourserver:2181

then do

ls /brokers/topics/

Do you see partition under this path ?

Dibyendu

Darpan Shah · Answer 4 · Wed Aug 24 2016 22:49:06 GMT+0800 (China Standard Time)

Yes, I can see the partitions. So my topic is metricsTopics. As you said, i can see the path: ls /brokers/topics/metricsTopics/paritions gives me [0,1,2,3,4,5] which is as expected because I said 6 partitions while creating. when I select one of them (i.e. 0) and do get cmnd, it displays information.
I saw my code again and I realize that i have changed zkRoot, and id in spoutConfig. So when I create an object of spoutConfig to create kafkaBolt, I am applying zkRoot as new SpoutConfig(hosts, "metricsTopics", "/metricsTopics", UUID.randomUUID().toString(). It was not UUID before. I dont remember what was it now. :(

Dibyendu Bhattacharya · Answer 5 · Wed Aug 24 2016 22:53:38 GMT+0800 (China Standard Time)

This consumer is for Spark Streaming, so spoutConfig is not applicable for Spark . You can probably raise this issue in Storm spout project.

Darpan Shah · Answer 6 · Wed Aug 24 2016 22:55:27 GMT+0800 (China Standard Time)

I am not using Spark though. In storm, I have configured zookeeper and kafka only. It was working before I restart my PC. I am trying to figure out how I can see the zkRoot of my topic so that I can replace that inside my spoutConfig code. Your help would be appreciated if you know this.

Dibyendu Bhattacharya · Answer 7 · Wed Aug 24 2016 23:00:18 GMT+0800 (China Standard Time)

You can probably go to /metricsTopics path in ZK to see what was set before

Darpan Shah · Answer 8 · Wed Aug 24 2016 23:08:22 GMT+0800 (China Standard Time)

Do you know where it should be? I checked under /tmp. There is not folder as 'Zookeeper', I checked zookeeper.properties file under kafka/config. it says datadir = /tmp/zookeeper. but its not there.
I checked data folder under zookeeper folder(where inside this, bin, config and all are there). Its not there.
But under /tmp/kafka-logs, I can see 6 folders named ;- metricsTopics-0, metricsTopics-1, ... till 6. Inside that , there is no log. They are empty.

Darpan Shah · Answer 9 · Thu Aug 25 2016 00:44:35 GMT+0800 (China Standard Time)

Working now! It was in zkroot. Well, first of all, I was connected to remote zookeeper, so I changed it to local. Second, I removed all the logs (/tmp/kafka-logs/ & /zookeeper/data/) and restart the zookeeper. It worked. I believe that its because of the remote zookeeper and over there topic was not created.

dhrubaroy11 · Answer 10 · Sun Sep 18 2016 19:41:56 GMT+0800 (China Standard Time)

HI Darpan,
I am having the same problem while running with remote zookeeper. With local zookeeper and kafka its running fine . So please let me know if you have any solution for running with remote zookeeper.