Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

limit the number of retries of the busy handler #963

Merged
merged 5 commits into from
Feb 1, 2024

Conversation

MarinPostma
Copy link
Contributor

@MarinPostma MarinPostma commented Jan 29, 2024

This PR fixes a series of bugs that caused hangs on the platform.

@MarinPostma MarinPostma enabled auto-merge January 29, 2024 16:22
@MarinPostma MarinPostma force-pushed the limit-busy-handler branch 3 times, most recently from e2035bd to ab00329 Compare January 30, 2024 19:08
let mut maybe_state_slot = self.state.slot.write();
// We need to make sure that the state slot is our slot before removing it.
if let Some(ref state_slot) = *maybe_state_slot {
if Arc::ptr_eq(state_slot, &old_slot) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a case where a pointer gets reused and we run into a ABA problem?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we discussed that last time, but we're actually comparing the pointer with itself, and we're still holding a reference to both ends while comparing, so no risk of it being reused in the meantime :)

@MarinPostma MarinPostma added this pull request to the merge queue Jan 30, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 30, 2024
Comment on lines 493 to 494
// We check that slot wasn't already stolen, and that their is still a slot.
// The ordering is relaxed because the atomic is only set under the slot lock.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment can probably go away, no?

Comment on lines +629 to +632
if !*is_stolen {
lock.rollback();
}
*is_stolen = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be in opposite order?

let stolen = *is_stolen
*is_stolen = true;
if !stolen {
  lock.rollback();
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, think of it as "if not already stolen, then rollback, then mark as stolen"

@MarinPostma MarinPostma force-pushed the limit-busy-handler branch 2 times, most recently from e6099f9 to 82c13b4 Compare January 31, 2024 10:31
Copy link
Collaborator

@psarna psarna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, later we can also push a low priority follow-up that brings back AtomicBool, since it's probably just as good -- and test it on canary

@MarinPostma MarinPostma added this pull request to the merge queue Feb 1, 2024
Merged via the queue into main with commit 7feeed5 Feb 1, 2024
12 checks passed
@MarinPostma MarinPostma deleted the limit-busy-handler branch February 1, 2024 09:21
LucioFranco pushed a commit that referenced this pull request Feb 7, 2024
* limit the amount of retries for the lock stealer

* use mutex for `is_stolen`

* unconditionally update state after execute

* shorten watch channel borrow

* fmt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants