How to List Available Kafka Brokers in a Cluster

In the world of distributed data streaming platforms, Apache Kafka has become the go-to choice for many developers and data engineers. It is a powerful tool that enables efficient handling of large volumes of data, making it an important component in modern data-driven applications.

If you're working with Kafka, chances are you'll need to manage and monitor your clusters, and one critical aspect of that process is knowing the available brokers within your cluster. In this article, we'll explore different ways to list those brokers, helping you keep tabs on your environment.

Understanding Kafka Brokers and Clusters

Before we get into how to list the available brokers, let's take a look at what exactly are brokers and clusters. Hopefully this will help you better understand the role of brokers in your Kafka setup.

The Role of Kafka Brokers

Kafka brokers are basically the backbone of any Kafka deployment. They are individual server instances that are responsible for receiving, storing, and transmitting messages within the cluster. They help to ensure fault tolerance and high availability by copying message data across multiple servers. By doing this, they play a critical role in maintaining the overall health of your Kafka system.

Every broker in the cluster is uniquely identified by an ID. This way clients and other brokers can locate and communicate with it. Brokers also handle tasks, such as message validation, compression, and indexing.

The Role of Kafka Clusters

A cluster is a group of brokers that work together to manage and distribute data. Clusters enable horizontal scalability and provide the fault tolerance necessary for building a robust, high-throughput system. They work with producers and consumers, which write and read messages, respectively, to and from the Kafka topics.

Clusters are responsible for managing data replication and making sure that messages are consistently available to consumers even if one or more brokers fail. They also handle load balancing, automatically redistributing data and processing loads among available brokers to maintain optimal performance.

Listing Kafka Brokers using Zookeeper

One of the ways to list available Kafka brokers in a cluster is by using the Zookeeper Shell. Zookeeper is a distributed coordination service that Kafka relies on for managing various aspects of its distributed nature, such as broker configuration, topic partitioning, and consumer group coordination. The Zookeeper Shell is a command-line interface that allows you to interact with a Zookeeper ensemble and perform various tasks, including listing the available Kafka brokers.

Here is how you would list the available brokers using the Zookeeper Shell:

  1. Make sure you have Kafka installed and open your terminal window.
  2. Navigate to the Kafka installation directory, typically found at /usr/local/kafka or /opt/kafka.
  3. Access the Zookeeper Shell by running the following command: ./bin/ localhost:2181. Make sure to replace localhost:2181 with the address and port of your Zookeeper ensemble if it is running on a different machine.
  4. Once you're connected to the shell, enter the following command to list the available brokers: ls /brokers/ids.
  5. The shell will display a list of available brokers, represented by their unique broker IDs.

The output you receive from the Zookeeper Shell will be a list of broker IDs, indicating the available Kafka brokers in your cluster. For example, if you see the following:

[1001, 1002, 1003]

It means that there are three available brokers with IDs 1001, 1002, and 1003.

Note: It's important to remember that the Zookeeper Shell only provides the broker IDs, not their addresses or other details. To get more information about a specific broker, you can then use the following command:

get /brokers/ids/<broker_id>

Just make sure to replace <broker_id> with the desired broker ID. This will return the broker's host, port, and other relevant details.

Now that we've covered how to list available Kafka brokers using the Zookeeper Shell, let's move on to another method: using Kafka CLI tools.

Listing Kafka Brokers using Kafka CLI Tools

In addition to the Zookeeper Shell, you can also use Kafka CLI tools to list available brokers in a cluster. Kafka ships with a number of CLI tools that can help manage and interact with a Kafka environment. For example, these tools can help you with things like creating and deleting topics, managing consumer groups, and retrieving broker info.

Here is how you'd list available brokers using the tool:

  1. Make sure you have Kafka installed and open your terminal window.
  2. Navigate to the Kafka install directory, typically found at /usr/local/kafka or /opt/kafka.
  3. Run the script with the --bootstrap-server option and followed by the address of one of your Kafka brokers. For example: ./bin/ --bootstrap-server localhost:9092. Again, replace localhost:9092 with whatever address and port your broker is running on.
  4. The script will connect to the specified broker and retrieve information about all available brokers in the cluster, including their IDs and other info.

The output of the script will provide you with a list of available brokers along with their supported API versions. Each broker will be listed on a line in the output, starting with the broker ID, the host and port, and supported API versions. For example:

Broker 1001 at localhost:9092 (controller): 0 to 2 [usable: 2], 1 to 3 [usable: 3], 3 to 5 [usable: 5], ...
Broker 1002 at localhost:9093: 0 to 2 [usable: 2], 1 to 3 [usable: 3], 3 to 5 [usable: 5], ...
Broker 1003 at localhost:9094: 0 to 2 [usable: 2], 1 to 3 [usable: 3], 3 to 5 [usable: 5], ...
Free eBook: Git Essentials

Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!

In this example, the output shows three available brokers with IDs 1001, 1002, and 1003 running on localhost with ports 9092, 9093, and 9094, respectively.

Automating with Scripts

While manually listing available Kafka brokers using the Zookeeper Shell or CLI tools is definitely helpful, automating the process can save you time and having to remember the commands. In this section, we'll show a simple example script that periodically lists available brokers using the tool.

Here's a simple Bash script that will list brokers every 10 minutes:



while true; do
    echo "Listing available brokers:"
    ${KAFKA_PATH}/bin/ --bootstrap-server ${BOOTSTRAP_SERVER}
    echo "Sleeping for ${INTERVAL} seconds..."
    sleep ${INTERVAL}

The output might look something like this:

$ ./
Listing available brokers:
Broker 1001 at localhost:9092 (controller): 0 to 2 [usable: 2], 1 to 3 [usable: 3], 3 to 5 [usable: 5], ...
Broker 1002 at localhost:9093: 0 to 2 [usable: 2], 1 to 3 [usable: 3], 3 to 5 [usable: 5], ...
Broker 1003 at localhost:9094: 0 to 2 [usable: 2], 1 to 3 [usable: 3], 3 to 5 [usable: 5], ...
Sleeping for 600 seconds...

You can likely think of any number of examples in which this can be used. The most important part of the script is the line starting with ${KAFKA_PATH}/bin/, which is where the actual call is made. Optionally, you could take this call, parse the output, and do something with the data. For example, you could use it to send notifications or log broker availability. Additionally, while this example uses, you could also modify it to use the Zookeeper Shell if needed.


In this article, we've explored two methods for listing available Kafka brokers in a cluster: using the Zookeeper Shell and Kafka CLI tools. We've also discussed automating the process with a simple script to periodically list brokers, which can make it easier to keep an eye on your Kafka environment. Regularly monitoring broker availability is crucial to maintaining the health, performance, and fault tolerance of your Kafka clusters, ensuring that your data streaming applications run smoothly and efficiently.

Last Updated: April 28th, 2023
Was this article helpful?

Improve your dev skills!

Get tutorials, guides, and dev jobs in your inbox.

No spam ever. Unsubscribe at any time. Read our Privacy Policy.

20% off

Hands On: Apache Kafka

# apache# Kafka

Don't miss out on our limited-time presale offer! For a limited time only, secure your spot in this comprehensive course at a special discounted price!...

Scott Robinson
David Landup
Arpendu Kumar Garai

Make Clarity from Data - Quickly Learn Data Visualization with Python

Learn the landscape of Data Visualization tools in Python - work with Seaborn, Plotly, and Bokeh, and excel in Matplotlib!

From simple plot types to ridge plots, surface plots and spectrograms - understand your data and learn to draw conclusions from it.

© 2013-2024 Stack Abuse. All rights reserved.