Don't add cluster to SM DB before validation finishes #4029
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Commit 5bf6b35 introduced a bug that when adding cluster to SM finished with error, cluster was still saved in SM DB, but only with the ID and known hosts fields. That's because method validateHostsConnectivity used discoverAndSetClusterHosts instead of discoverClusterHosts, which happened before inserting cluster to SM DB. Fixes #4028
Michal-Leszczynski
force-pushed
the
ml/fix-4028
branch
from
62c01b9 to
9356865
Compare
E.g. clusters created before SM 3.2.6 can have empty --host SM DB column.
In case client was mistakenly initialized with "" host, getting host from the pool would panic: panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x11da5ac] goroutine 253 [running]: github.com/scylladb/scylla-manager/v3/pkg/scyllaclient.NewClient.hostPool.func3(0xc0004345a0) github.com/scylladb/scylla-manager/v3/pkg/scyllaclient/hostpool.go:32 +0xac This should be replaced with a simple error. This commit adds additional validation to client config, which prohibits specifying empty host there. It also makes host pool return an error instead of panic in case of invalid host pool response. Fixes scylladb/scylla-enterprise#4692
As it is possible that the --host field in SM DB is empty, we should test that this scenario does not break anything in SM. Tests scylladb/scylla-enterprise#4692
Michal-Leszczynski
force-pushed
the
ml/fix-4028
branch
from
9356865 to
0008113
Compare
|
@karol-kokoszka This PR is ready for review! Failures are connected to #4032. |
|
Thanks ! Scylla-manager started with ... what is fine. Eventually, it is possible to update the host with |
|
@Michal-Leszczynski can you run operator tests on that ? |
|
Operator tests passed, but since this time I added some more safety nets and tests, so I will re-trigger them. |
|
Operator tests started here: scylladb/scylla-operator#2105 (comment). |
|
@karol-kokoszka operator tests passed: scylladb/scylla-operator#2105 (comment)! |
|
OK, so merging to master and backporting to branch-3.3 |
karol-kokoszka
approved these changes
Sep 11, 2024
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Commit 5bf6b35 introduced a bug that when adding cluster to SM finished with error,
cluster was still saved in SM DB, but only with the ID and known hosts fields.
That's because method validateHostsConnectivity used discoverAndSetClusterHosts
instead of discoverClusterHosts, which happened before inserting cluster to SM DB.
In case client was mistakenly initialized with "" host, getting host from the pool would panic.
This should be replaced with a simple error. This PR adds additional validation to client config,
which prohibits specifying empty host there. It also makes host pool return an error instead of panic
in case of invalid host pool response.
This PR also includes testes for mentioned issues (validated that those tests weren't passing without the fixes).
Fixes #4028