Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple compaction commit from reads #90

Open
marvin-j97 opened this issue Dec 7, 2024 · 0 comments
Open

Decouple compaction commit from reads #90

marvin-j97 opened this issue Dec 7, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed performance refactor test

Comments

@marvin-j97
Copy link
Contributor

marvin-j97 commented Dec 7, 2024

Right now when a compaction wants to commit its changes, it needs to write lock the active memtable to make sure that it cannot drop files that may still be read currently.
Instead, it should only mark segments as deleted (atomic bool), so it does not need the memtable lock at all.
On drop, the segments should delete themselves.
That way, when a read starts, it keeps holding its referenced segments alive; this may increase disk space usage temporarily, but that's fine.

@marvin-j97 marvin-j97 added enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed performance refactor test labels Dec 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed performance refactor test
Projects
None yet
Development

No branches or pull requests

1 participant