-
Notifications
You must be signed in to change notification settings - Fork 398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Internal] Restart Cluster before Library Installation #4384
base: main
Are you sure you want to change the base?
Conversation
If integration tests don't run automatically, an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
@parthban-db we have #4036 that mentions that we can install libraries in UI without starting the cluster - the question is if it works via API the same way or not... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general is good, see my comment about existing issue about installing on terminated clusters via UI
I will ask the team about this. |
@alexott I talked with the team, and according to them, |
@parthban-db, We can make it controllable by the user - we start the cluster and install the library just to check if there are no errors. But if we can allow users to specify that they don't want to wait for it, then it could be a solution. @mgyucht wdyt? |
LGTM: we revert this back to the original & documented behavior of starting the cluster if it isn't already running. Longer term, we can revisit how Terraform users will think about cluster and library lifecycles. I can see this as very useful in some use-cases (e.g. DABs), but there may also be users who don't want a cluster to be started for the sake of creating/updating a library. |
PreConfig: func() { | ||
// Delete the created cluster | ||
w := databricks.Must(databricks.NewWorkspaceClient()) | ||
getter, err := w.Clusters.Delete(context.Background(), compute.DeleteCluster{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What a confusing name for this method... @renaudhartert-db maybe we want to revisit this in the next version of the SDK. It should really be "terminate cluster" to align with basically all other terminology.
Changes
Library resource with SDK_V2 implementation waits for the cluster to get into the running state before starting installation. This PR mimics this behavior in the plugin framework implementation of the library resource.
Fixes #4353
Tests
Added a integration test.
make test
run locallydocs/
folderinternal/acceptance