Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add mutex to internal trace otel test variable #9390

Merged
merged 3 commits into from
Feb 8, 2024

Conversation

harshachinta
Copy link
Contributor

@harshachinta harshachinta commented Feb 8, 2024

Fixes #9388

The variable OpenTelemetryTracingEnabled is set by client library for testing purposes. However, there is a datarace condition that occurs when this variable is getting read in trace package and set in client library.
Sigma link
Interestingly this is not reproducible on local with -race argument and not in presubmits too. But it failed in continuous build.

Stack trace

=== RUN   TestSpannerTracesWithOpenTelemetry
==================
WARNING: DATA RACE
Write at 0x0000026759b0 by goroutine 536:
  cloud.google.com/go/spanner/test.TestSpannerTracesWithOpenTelemetry()
      /tmpfs/src/google-cloud-go/spanner/test/opentelemetry/test/ot_traces_test.go:37 +0x6b
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1595 +0x238
  testing.(*T).Run.func1()
      /usr/local/go/src/testing/testing.go:1648 +0x44

Previous read at 0x0000026759b0 by goroutine 500:
  cloud.google.com/go/internal/trace.TracePrintf()
      /tmpfs/src/google-cloud-go/internal/trace/trace.go:190 +0x72
  cloud.google.com/go/spanner.(*sessionClient).executeBatchCreateSessions()
      /tmpfs/src/google-cloud-go/spanner/sessionclient.go:310 +0x1c5a
  cloud.google.com/go/spanner.(*sessionClient).batchCreateSessions.func2()
      /tmpfs/src/google-cloud-go/spanner/sessionclient.go:234 +0x8c

Goroutine 536 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:1648 +0x82a
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:2054 +0x84
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1595 +0x238
  testing.runTests()
      /usr/local/go/src/testing/testing.go:2052 +0x896
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1925 +0xb57
  main.main()
      _testmain.go:57 +0x2bd

Goroutine 500 (finished) created at:
  cloud.google.com/go/spanner.(*sessionClient).batchCreateSessions()
      /tmpfs/src/google-cloud-go/spanner/sessionclient.go:234 +0x4ec
  cloud.google.com/go/spanner.(*sessionPool).growPoolLocked()
      /tmpfs/src/google-cloud-go/spanner/session.go:812 +0x187
  cloud.google.com/go/spanner.(*sessionPool).take()
      /tmpfs/src/google-cloud-go/spanner/session.go:1074 +0x788
  cloud.google.com/go/spanner.(*ReadOnlyTransaction).acquireSingleUse()
      /tmpfs/src/google-cloud-go/spanner/transaction.go:803 +0x553
  cloud.google.com/go/spanner.(*ReadOnlyTransaction).acquire()
      /tmpfs/src/google-cloud-go/spanner/transaction.go:780 +0x7b
  cloud.google.com/go/spanner.(*txReadOnly).ReadWithOptions()
      /tmpfs/src/google-cloud-go/spanner/transaction.go:228 +0x276
  cloud.google.com/go/spanner.(*txReadOnly).Read()
      /tmpfs/src/google-cloud-go/spanner/transaction.go:148 +0xb04
  cloud.google.com/go/spanner/test.TestOTMetrics_GFELatency()
      /tmpfs/src/google-cloud-go/spanner/test/opentelemetry/test/ot_metrics_test.go:288 +0xa4f
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1595 +0x238
  testing.(*T).Run.func1()
      /usr/local/go/src/testing/testing.go:1648 +0x44
==================
    testing.go:1465: race detected during execution of test
--- FAIL: TestSpannerTracesWithOpenTelemetry (0.02s)
=== NAME  
    testing.go:1465: race detected during execution of test

@harshachinta harshachinta requested review from a team as code owners February 8, 2024 14:43
@product-auto-label product-auto-label bot added the size: s Pull request size is small. label Feb 8, 2024
internal/trace/trace.go Outdated Show resolved Hide resolved
spanner/test/opentelemetry/test/ot_traces_test.go Outdated Show resolved Hide resolved
@harshachinta harshachinta requested a review from quartzmo February 8, 2024 16:39
internal/trace/trace.go Outdated Show resolved Hide resolved
internal/trace/trace.go Outdated Show resolved Hide resolved
@harshachinta harshachinta changed the title fix: add mutex to otel test variable fix: add mutex to internal trace otel test variable Feb 8, 2024
@harshachinta harshachinta merged commit 909fbdf into googleapis:main Feb 8, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: s Pull request size is small.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

spanner/test: TestSpannerTracesWithOpenTelemetry failed
3 participants