How do I manage consumer groups in Kafka using Python?

How do I manage consumer groups in Kafka using Python?

You can manage consumer groups in Kafka using the kafka-python library. Would you like to know how to create, list, or delete consumer groups?

I'd like to know how to list consumer groups.

To list consumer groups, you can use the AdminClient class from kafka-python. Here's a sample code snippet:

from kafka import KafkaAdminClient

admin_client = KafkaAdminClient(bootstrap_servers='localhost:9092')
consumer_groups = admin_client.list_consumer_groups()
print(consumer_groups)

What about creating a new consumer group?

Creating a new consumer group is done automatically when you assign a new group ID to a consumer. Just set the group_id parameter when creating your consumer instance.

Can you show me an example of creating a consumer?

Sure! Here's an example of creating a Kafka consumer with a specific group ID:

from kafka import KafkaConsumer

consumer = KafkaConsumer('my_topic', group_id='my_group', bootstrap_servers='localhost:9092')
for message in consumer:
    print(message.value)