Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Load balancing cache in POP consumption mode #9131

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

YoWuwuuuw
Copy link

Which Issue(s) This PR Fixes

Add cache for each client request broker end rebalancing in the pop consumption mode to avoid wasting computing resources of broker end due to repeated calculation of load balancing without changes. The effect is more obvious in theory when there are a large number of queues or consumers

Fixes #9123

Brief Description

To add caching for broker-end rebalancing under the pop consumption model, use ReentrantLock to maintain consistency and efficiency. Before performing load balancing calculations, attempt to obtain cache entries when the client list and messagequeue list remain unchanged. Only if the cache is not hit will the calculation logic be executed, using minimal space to avoid wasting broker computing resources and ensuring efficient rebalancing requests

How Did You Test This Change?

Start a name serve, a broker, and two consumer clients locally, using the pop mode for consumption.

Test the performance of broker-side rebalancing with both the original version and the improved version, covering:

  1. consumers default to requesting rebalancing results every 20 seconds
  2. active request rebalancing results when the consumer group changes
  3. default request rebalancing results when the topic queue changes

@3424672656
Copy link
Contributor

Hello, please fix the code running test and output the test document

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] Load balancing cache in POP consumption mode
2 participants