Obtaining topic metadata

[Michael Graff]

I would like to use Poseidon as part of a Kafka monitoring tool. This means I'd like to get the topic metadata periodically, which I hope includes the first and last offset on the topic, the current leader, and the ISR status.

Ideally, this would also have a way to list the topics as well.

Is this possible with the current code, and if so, how? If not, what approach would you use?

Thanks!

[Andrew Hammond]

Here is how I do it:

Poseidon::BrokerPool.open client_id, seed_brokers, (kafka_settings[:socket_timeout_ms] || 1_000) do |broker_pool|
  cluster_metadata = Poseidon::ClusterMetadata.new

  # you could pass multiple partitions, in my case i'm looking for a specific one
  cluster_metadata.update broker_pool.fetch_metadata [topic]
  topics_metadata = cluster_metadata.metadata_for_topics [topic]
  metadata = topics_metadata[topic]

  partitions = metadata.available_partitions
  # ... do stuff with partitions
end