Breaking News: Grepper is joining You.com. Read the official announcement!

What are consumers in Apache Kafka?

Pragya Keshap answered on February 6, 2023 Popularity 2/10 Helpfulness 1/10

answer What are consumers in Apache Kafka?

What are consumers in Apache Kafka?

Comment

Kafka consumers act as end-users or applications that retrieve data from Kafka servers inside which Kafka producers publish real-time messages.

For effectively fetching real-time messages, Kafka consumers have to subscribe to the respective topics present inside the Kafka servers.

private Map<TopicPartition, OffsetAndMetadata> currentOffsets =
    new HashMap<>();

private class HandleRebalance implements ConsumerRebalanceListener { 
    public void onPartitionsAssigned(Collection<TopicPartition>
        partitions) { 
    }

    public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
        System.out.println("Lost partitions in rebalance. " +
            "Committing current offsets:" + currentOffsets);
        consumer.commitSync(currentOffsets); 
    }
}

try {
    consumer.subscribe(topics, new HandleRebalance()); 

    while (true) {
        ConsumerRecords<String, String> records = consumer.poll(100);
        for (ConsumerRecord<String, String> record : records) {
            System.out.printf("topic = %s, partition = %s, offset = %d,
                 customer = %s, country = %s\n",
                 record.topic(), record.partition(), record.offset(),
                 record.key(), record.value());
             currentOffsets.put(
                 new TopicPartition(record.topic(), record.partition()),
                 new OffsetAndMetadata(record.offset()+1, null));
        }
        consumer.commitAsync(currentOffsets, null);
    }
} catch (WakeupException e) {
    // ignore, we're closing
} catch (Exception e) {
    log.error("Unexpected error", e);
} finally {
    try {
        consumer.commitSync(currentOffsets);
    } finally {
        consumer.close();
        System.out.println("Closed consumer and we are done");
    }
}

https://learning.oreilly.com/library/view/kafka-the-definitive/9781491936153/ch04.html#idm45788273460648

Popularity 2/10 Helpfulness 1/10 Language java

Source: Grepper

Tags: java

Link to this answer
Share Copy Link

Contributed on Feb 06 2023

Pragya Keshap

0 Answers Avg Quality 2/10

What are consumers in Apache Kafka?

Contents

More Related Answers

What are consumers in Apache Kafka?

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.