[improve] [pip] PIP-358: let resource weight work for OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl. #22889
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thetumbled
changed the title
15 tasks
|
PTAL, thanks. @Demogorgon314 @heesung-sn @codelipenghui @BewareMyPower @Technoboy- |
|
Could you clarify what's the detail update for OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImp by this PIP? |
Updated, PTAL, thanks. |
Demogorgon314
approved these changes
Jun 13, 2024
heesung-sn
approved these changes
Jun 13, 2024
Technoboy-
approved these changes
Jun 14, 2024
hanmz
pushed a commit
to hanmz/pulsar
that referenced
this pull request
Feb 12, 2025
…, LeastLongTermMessageRate, ModularLoadManagerImpl. (apache#22889) Implementation PR: apache#22888 ### Motivation Initially, we introduce `loadBalancerCPUResourceWeight`, `loadBalancerBandwidthInResourceWeight`, `loadBalancerBandwidthOutResourceWeight`, `loadBalancerMemoryResourceWeight`, `loadBalancerDirectMemoryResourceWeight` in `ThresholdShedder` to control the resource weight for different resources when calculating the load of the broker. Then we let it work for `LeastResourceUsageWithWeight` for better bundle placement policy. But apache#19559 and apache#21168 have point out that the actual load of the broker is not related to the memory usage and direct memory usage, thus we have changed the default value of `loadBalancerMemoryResourceWeight`, `loadBalancerDirectMemoryResourceWeight` to 0.0. There are still some places where memory usage and direct memory usage are used to calculate the load of the broker, such as `OverloadShedder`, `LeastLongTermMessageRate`, `ModularLoadManagerImpl`. We should let the resource weight work for these places so that we can set the resource weight to 0.0 to avoid the impact of memory usage and direct memory usage on the load of the broker. ### Modifications - Let resource weight work for `OverloadShedder`, `LeastLongTermMessageRate`, `ModularLoadManagerImpl`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implementation PR: #22888
Motivation
Initially, we introduce
loadBalancerCPUResourceWeight,loadBalancerBandwidthInResourceWeight,loadBalancerBandwidthOutResourceWeight,loadBalancerMemoryResourceWeight,loadBalancerDirectMemoryResourceWeightinThresholdShedderto control the resource weight for different resources when calculating the load of the broker.Then we let it work for
LeastResourceUsageWithWeightfor better bundle placement policy.But #19559 and #21168 have point out that the actual load of the broker is not related to the memory usage and direct memory usage, thus we have changed the default value of
loadBalancerMemoryResourceWeight,loadBalancerDirectMemoryResourceWeightto 0.0.There are still some places where memory usage and direct memory usage are used to calculate the load of the broker, such as
OverloadShedder,LeastLongTermMessageRate,ModularLoadManagerImpl. We should let the resource weight work for these places so that we can set the resource weight to 0.0 to avoid the impact of memory usage and direct memory usage on the load of the broker.Modifications
OverloadShedder,LeastLongTermMessageRate,ModularLoadManagerImpl.Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: