Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bigtable/bttest): Implement change streams #9203

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

adg
Copy link
Contributor

@adg adg commented Dec 28, 2023

These changes add an implementation of the GenerateInitialChangeStreamPartitions and ReadChangeStream BigtableServer methods to the bttest package, and also fix a bug related to commit times.

@adg adg requested review from igorbernstein2 and a team as code owners December 28, 2023 00:48
@product-auto-label product-auto-label bot added the size: l Pull request size is large. label Dec 28, 2023
adg added 2 commits December 28, 2023 13:42
This is an important semantic property of BigTable:

  "All mutations in a single write request have the same timestamp
  unless you override them."

  https://cloud.google.com/bigtable/docs/writes#write-types
Implement the GenerateInitialChangeStreamPartitions and ReadChangeStream
BigtableServer methods, and record and serve change streams for tables
that are created with a ChangeStreamConfig specified.

Note: this change does not implement sending Heartbeat records from
ReadChangeStream, and the retention period specified by the client is
not honored. These are left as TODOs.
@adg
Copy link
Contributor Author

adg commented Dec 28, 2023

I'm not sure why those conformance tests are failing. Any pointers?

@codyoss codyoss changed the title Implement change streams in bttest feat(bigtable/bttest): Implement change streams Jan 1, 2024
@product-auto-label product-auto-label bot added the api: bigtable Issues related to the Bigtable API. label Jan 2, 2024
@bhshkh
Copy link
Contributor

bhshkh commented Jan 19, 2024

The conformance tests were added in failed state. Their failures can be ignored

@bhshkh
Copy link
Contributor

bhshkh commented Jan 19, 2024

What is the bug related to commit times?

@product-auto-label product-auto-label bot added the stale: old Pull request is old and needs attention. label Jan 27, 2024
@product-auto-label product-auto-label bot added stale: extraold Pull request is critically old and needs prioritization. and removed stale: old Pull request is old and needs attention. labels Feb 26, 2024
@adg
Copy link
Contributor Author

adg commented Apr 17, 2024

@bhshkh to quote the commit message:

This is an important semantic property of BigTable:

"All mutations in a single write request have the same timestamp
unless you override them."

https://cloud.google.com/bigtable/docs/writes#write-types

@adg
Copy link
Contributor Author

adg commented Apr 17, 2024

Our use-case for this is that we have a service that uses change streams and we want to be able to test it locally.

@bhshkh bhshkh requested a review from a team as a code owner July 17, 2024 00:39
@bhshkh bhshkh enabled auto-merge (squash) July 18, 2024 00:23
@bhshkh
Copy link
Contributor

bhshkh commented Jul 18, 2024

@adg , can you please sign the CLA to merge the PR. please visit https://cla.developers.google.com/

@codyoss
Copy link
Member

codyoss commented Dec 10, 2024

@adg can you please sign the CLA if you would still like these changes, or else we can close this PR if you are no longer interested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the Bigtable API. size: l Pull request size is large. stale: extraold Pull request is critically old and needs prioritization.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants