Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] KafkaJS handling of failed heartbeat calls #1719

Open
krumft opened this issue Nov 4, 2024 · 1 comment
Open

[Question] KafkaJS handling of failed heartbeat calls #1719

krumft opened this issue Nov 4, 2024 · 1 comment

Comments

@krumft
Copy link

krumft commented Nov 4, 2024

Hi Team,

I am opening this issue just for the sake of asking a question. When the KafkaJS library performs a heartbeat call to the Kafka cluster, and receives an error that the consumer group is being rebalanced:

  1. will KafkaJS pause the consuming for that consumer? (and presumably resume it when this consumer successfully rejoins the group) - this would make sense as it is no longer clear for which topic partitions the consumer is responsible
  2. will KafkaJS start preparing the consumer to issue a ReJoin request to Kafka?

Looking at https://github.com/tulios/kafkajs/blob/master/src/consumer/runner.js#L207-L216 it seems that neither of these two happen.

The reasons behind these questions of mine are the following two:

  1. We are seeing evidence of both 1. and 2. happening in the case of the Java consumer;
  2. If we manually implement 1. and 2. to happen during our application-level handling of heartbeat it seems we can solve some nasty issues of constantly seeing rebalancing.

Thanks a lot for your time,
Krum.

@krumft
Copy link
Author

krumft commented Nov 4, 2024

Related question: #1645

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant