Breaking News: Grepper is joining You.com. Read the official announcement!
Check it out

Why we need consumer parallel processing?

Sumit Rawal answered on May 31, 2023 Popularity 1/10 Helpfulness 1/10

Contents


More Related Answers


Why we need consumer parallel processing?

0

If you have a topic with four partitions and only one consumer in a group, that consumer would consume records from all the partitions.

This may work for you in some cases, but what if the rate at which producers write messages to the topic exceeds the rate at which your consumer application can process them? If you are still limited to a single consumer reading and processing the data, your application may be unable to keep up with the rate of incoming messages. In Kinesis, this latency is called iterator age. Obviously there is a need to scale consumer consumption from topics. Just like multiple producers can write to the same topic, we need a similar mechanism to allow multiple consumers to read from the same topic, sharing the data between them.

Popularity 1/10 Helpfulness 1/10 Language whatever
Source: Grepper
Link to this answer
Share Copy Link
Contributed on May 31 2023
Sumit Rawal
0 Answers  Avg Quality 2/10


X

Continue with Google

By continuing, I agree that I have read and agree to Greppers's Terms of Service and Privacy Policy.
X
Grepper Account Login Required

Oops, You will need to install Grepper and log-in to perform this action.