feat(storage): merge support for bi-directional multiple range reads #11377
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
product-auto-label
bot
added
the
api: storage
cojenco
added
the
kokoro:force-run
kokoro-team
removed
the
kokoro:force-run
danielduhh
approved these changes
Jan 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge the following commits:
feat(storage): add MultiRangeDownloader
feat(storage): add support for Read handle cache
feat(storage): fix panic raised due to reader.ReadHandle()
test(storage): use gcs bucket for cloud build logs
fix: Use run's Context in gRPCWriter
feat(storage): add emulator tests
feat(append): Import new proto patch.
feat(append): Refactor gRPCWriter flow.
feat(append): Initial append writer.
feat(append): Allow retries with preconditions.
feat(append): Update Object with finalize_time.
feat(append): Appendable object retry tests.
feat(append): Support redirection errors.
feat(storage): make gRPC Reader use Bidi API
feat(storage): support ReadHandle for Reader
feat(storage): add sanity checks in gRPC BidiReader.
chore: update tests to close reader
fix: Handle generation in Bidi NewReader.
fix: Client half-close on oneshot reads.
fix: Set appendable=true in outbound metadata.
fix: Correct the bucket in append writer header.
fix: update error logic and add integration test for MRD
fix: remove race condition in MRD and add retry conf tests
fix: Don't reuse a stream after CloseSend().
feat(storage): add a check for first range which can succed or fail.
feat(storage): fix concurrent write on map in multirangedownloader
feat(storage): refactor multirangedownloader
feat(storage): support ReadObject
fix(storage): add backoff to upload retries
chore: Update bidi proto patch.
fix(storage): fix offset and remaining bytes calculation
fix(storage): use mutex for accessing concurrent vars