Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Internal] Migrate Share resource to Go SDK #3916

Merged
merged 14 commits into from
Sep 3, 2024
Merged

[Internal] Migrate Share resource to Go SDK #3916

merged 14 commits into from
Sep 3, 2024

Conversation

rauchy
Copy link
Contributor

@rauchy rauchy commented Aug 19, 2024 •
edited
Loading

Changes

This PR is my first attempt at migrating the Share resource from using hand-crafted API calls to the Databricks Go SDK. I’ve updated the Read function to use the SDK’s Get method. Next, I’ll work on migrating the remaining CRUD operations. I’ve run the existing tests to ensure everything is still working as expected.

Update: also included Create & Update here, which makes the Share resource go through the Go SDK for all CRUD operations.

Tests

  • make test run locally
  • relevant change in docs/ folder
  • covered with integration tests in internal/acceptance
  • relevant acceptance tests are passing
  • using Go SDK

@rauchy rauchy requested a review from tanmay-db August 19, 2024 15:04
@rauchy rauchy requested review from a team as code owners August 19, 2024 15:04
@rauchy rauchy changed the title Migrate Share resource to Go SDK Migrate Read Share resource to Go SDK Aug 19, 2024
@rauchy rauchy force-pushed the migrate-resource-share-to-go-sdk branch from aef360c to 65197d9 Compare August 20, 2024 07:06
@rauchy rauchy changed the title Migrate Read Share resource to Go SDK [Internal] Migrate Read Share resource to Go SDK Aug 20, 2024
@rauchy rauchy force-pushed the migrate-resource-share-to-go-sdk branch from 65197d9 to 49f6bba Compare August 20, 2024 07:09
tanmay-db
tanmay-db reviewed Aug 20, 2024
View reviewed changes
Copy link
Contributor

@tanmay-db tanmay-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, looks great, just had a few comments. I have triggered nightly test run.

sharing/data_share.go Outdated
CreatedAt int64 `json:"created_at,omitempty" tf:"computed"`
CreatedBy string `json:"created_by,omitempty" tf:"computed"`
Name string `json:"name,omitempty" tf:"computed"`
Objects []sharing.SharedDataObject `json:"objects,omitempty" tf:"computed,slice_set,alias:object"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes would be required before we can do this, for example:
StartVersion in SharedDataObject is annotated tf:"suppress_diff" whereas it is not in Go SDK struct.

Also, DataObjectType is string in SharedDataObject but SharedDataObjectDataObjectType in Go SDK which I think is okay but needs to be verified if this doesn't cause any other issue (eg: if its used in test, then that would need to be upgraded, if it's not in test, might be good to add a test).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed in 20b08cb

sharing/resource_share.go
@@ -208,17 +206,28 @@ func ResourceShare() common.Resource {
return nil
},
Read: func(ctx context.Context, d *schema.ResourceData, c *common.DatabricksClient) error {
si, err := NewSharesAPI(ctx, c).get(d.Id())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

si.sortSharesByName()
si.suppressCDFEnabledDiff()

would also be needed here as we were doing it before + would be required before we do StructToData below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 910decf

@rauchy rauchy force-pushed the migrate-resource-share-to-go-sdk branch 3 times, most recently from 64729a1 to c53a84e Compare August 27, 2024 10:49
@rauchy rauchy changed the title [Internal] Migrate Read Share resource to Go SDK [Internal] Migrate Share resource to Go SDK Aug 27, 2024
@rauchy rauchy force-pushed the migrate-resource-share-to-go-sdk branch 6 times, most recently from d647e61 to fdef29d Compare August 28, 2024 14:26
@rauchy rauchy force-pushed the migrate-resource-share-to-go-sdk branch from fdef29d to 90319a2 Compare September 2, 2024 09:59
@rauchy rauchy added this pull request to the merge queue Sep 3, 2024
Merged via the queue into main with commit 661e682 Sep 3, 2024
7 checks passed
@rauchy rauchy deleted the migrate-resource-share-to-go-sdk branch September 3, 2024 09:29
@hectorcast-db hectorcast-db mentioned this pull request Sep 12, 2024
Closed
hectorcast-db added a commit that referenced this pull request Sep 12, 2024
@hectorcast-db
[Release] Release v1.52.0
5206f57 
### New Features and Improvements

 * Added `no_wait` option for clusters to skip waiting to start on cluster creation ([#3953](#3953)).
 * Introduced Plugin Framework ([#3920](#3920)).

### Bug Fixes

 * Add suppress diff for `azure_attributes.spot_bid_max_price` in `databricks_instance_pool` ([#3970](#3970)).
 * Correctly send workload_type fields in `databricks_cluster` to allow users to disable usage in certain contexts ([#3972](#3972)).
 * Fix `TestAccClusterResource_WorkloadType` ([#3989](#3989)).
 * Fix `databricks_sql_table` treatment of properties ([#3925](#3925)).
 * Force send fields for settings resources ([#3978](#3978)).
 * Handle cluster deletion in `databricks_library` read ([#3909](#3909)).
 * Make subscriptions optional for SqlAlertTask ([#3983](#3983)).

### Documentation

 * Add troubleshooting guide for Provider "registry.terraform.io/databricks/databricks" planned an invalid value ([#3961](#3961)).
 * Adopt official naming of Mosaic AI Vector Search ([#3971](#3971)).
 * Document Terraform 1.0 as minimum version ([#3952](#3952)).
 * Mention Salesforce as supported type in `databricks_connection` ([#3949](#3949)).
 * Reimplement Azure Databricks deployment guide to use VNet injection & NPIP ([#3986](#3986)).
 * Resolves [#3127](#3127): Remove deprecated account_id field from mws_credentials resource ([#3974](#3974)).
 * Small Grammar Corrections in Docs ([#4006](#4006)).
 * Update `databricks_vector_search_index` docs to match latest SDK ([#4008](#4008)).
 * Update aws_unity_catalog_assume_role_policy.md ([#3968](#3968)).
 * Update documentation regarding authentication with Azure-managed Service Principal using GITHUB OIDC ([#3932](#3932)).
 * Update metastore_assignment.md to properly reflect possible usage ([#3967](#3967)).
 * Update minimum supported terraform version to 1.1.5 ([#3965](#3965)).
 * Update resources diagram to include newer resources ([#3962](#3962)).
 * Update workspace_binding import command ([#3944](#3944)).
 * fix possible values for `securable_type` in `databricks_workspace_binding` ([#3942](#3942)).

### Internal Changes

 * Add `AddPlanModifer` method for AttributeBuilder ([#4009](#4009)).
 * Add integration tests for volumes and quality monitor plugin framework ([#3975](#3975)).
 * Add support for `computed` tag in TfSDK Structs ([#4005](#4005)).
 * Added `databricks_quality_monitor` resource and `databricks_volumes` data source to plugin framework ([#3958](#3958)).
 * Allow vector search tests to fail ([#3959](#3959)).
 * Make test utils public and move integration test for quality monitor ([#3993](#3993)).
 * Migrate Share resource to Go SDK ([#3916](#3916)).
 * Migrate imports for terraform plugin framework + update init test provider factory ([#3943](#3943)).
 * Move volumes test next to plugin framework data source ([#3995](#3995)).
 * Refactor provider and related packages ([#3940](#3940)).
 * Support import in acceptance test + adding import state for quality monitor ([#3994](#3994)).

### Dependency Updates

 * Bump github.com/hashicorp/hcl/v2 from 2.21.0 to 2.22.0 ([#3948](#3948)).
 * Update Go SDK to 0.46.0 ([#4007](#4007)).

### Exporter

 * Don't generate instance pools if the pool name is empty ([#3960](#3960)).
@tanmay-db tanmay-db mentioned this pull request Sep 16, 2024
Merged
tanmay-db added a commit that referenced this pull request Sep 16, 2024
@tanmay-db
[Release] Release v1.52.0
c83c514 
### New Features and Improvements

 * Add support for filters in `databricks_clusters` data source ([#4014](#4014)).
 * Added `no_wait` option for clusters to skip waiting to start on cluster creation ([#3953](#3953)).
 * Introduced Plugin Framework ([#3920](#3920)).
 * Library plugin framework migration ([#3979](#3979)).

### Bug Fixes

 * Add suppress diff for `azure_attributes.spot_bid_max_price` in `databricks_instance_pool` ([#3970](#3970)).
 * Correctly send workload_type fields in `databricks_cluster` to allow users to disable usage in certain contexts ([#3972](#3972)).
 * Fix `TestAccClusterResource_WorkloadType` ([#3989](#3989)).
 * Fix `databricks_sql_table` treatment of properties ([#3925](#3925)).
 * Force send fields for settings resources ([#3978](#3978)).
 * Handle cluster deletion in `databricks_library` read ([#3909](#3909)).
 * Make subscriptions optional for SqlAlertTask ([#3983](#3983)).
 * Permanently delete `ERROR` and `TERMINATED` state clusters if their creation fails ([#4021](#4021)).

### Documentation

 * Add troubleshooting guide for Provider "registry.terraform.io/databricks/databricks" planned an invalid value ([#3961](#3961)).
 * Adopt official naming of Mosaic AI Vector Search ([#3971](#3971)).
 * Document Terraform 1.0 as minimum version ([#3952](#3952)).
 * Mention Salesforce as supported type in `databricks_connection` ([#3949](#3949)).
 * Reimplement Azure Databricks deployment guide to use VNet injection & NPIP ([#3986](#3986)).
 * Resolves [#3127](#3127): Remove deprecated account_id field from mws_credentials resource ([#3974](#3974)).
 * Small Grammar Corrections in Docs ([#4006](#4006)).
 * Update `databricks_vector_search_index` docs to match latest SDK ([#4008](#4008)).
 * Update aws_unity_catalog_assume_role_policy.md ([#3968](#3968)).
 * Update documentation regarding authentication with Azure-managed Service Principal using GITHUB OIDC ([#3932](#3932)).
 * Update metastore_assignment.md to properly reflect possible usage ([#3967](#3967)).
 * Update minimum supported terraform version to 1.1.5 ([#3965](#3965)).
 * Update resources diagram to include newer resources ([#3962](#3962)).
 * Update workspace_binding import command ([#3944](#3944)).
 * fix possible values for `securable_type` in `databricks_workspace_binding` ([#3942](#3942)).

### Internal Changes

 * Add `AddPlanModifer` method for AttributeBuilder ([#4009](#4009)).
 * Add integration tests for volumes and quality monitor plugin framework ([#3975](#3975)).
 * Add support for `computed` tag in TfSDK Structs ([#4005](#4005)).
 * Added `databricks_quality_monitor` resource and `databricks_volumes` data source to plugin framework ([#3958](#3958)).
 * Allow vector search tests to fail ([#3959](#3959)).
 * Clean up comments in library resource ([#4015](#4015)).
 * Fix irregularities in plugin framework converter function errors ([#4010](#4010)).
 * Make test utils public and move integration test for quality monitor ([#3993](#3993)).
 * Migrate Share resource to Go SDK ([#3916](#3916)).
 * Migrate `databricks_cluster` data source to plugin framework ([#3988](#3988)).
 * Migrate imports for terraform plugin framework + update init test provider factory ([#3943](#3943)).
 * Move volumes test next to plugin framework data source ([#3995](#3995)).
 * Refactor provider and related packages ([#3940](#3940)).
 * Support import in acceptance test + adding import state for quality monitor ([#3994](#3994)).

### Dependency Updates

 * Bump github.com/hashicorp/hcl/v2 from 2.21.0 to 2.22.0 ([#3948](#3948)).
 * Update Go SDK to 0.46.0 ([#4007](#4007)).

### Exporter

 * Don't generate instance pools if the pool name is empty ([#3960](#3960)).
github-merge-queue bot pushed a commit that referenced this pull request Sep 17, 2024
@tanmay-db
[Release] Release v1.52.0 (#4022)
e13ce2c 
### New Features and Improvements

* Add support for filters in `databricks_clusters` data source
([#4014](#4014)).
* Added `no_wait` option for clusters to skip waiting to start on
cluster creation
([#3953](#3953)).
* Introduced Plugin Framework
([#3920](#3920)).


### Bug Fixes

* Add suppress diff for `azure_attributes.spot_bid_max_price` in
`databricks_instance_pool`
([#3970](#3970)).
* Correctly send workload_type fields in `databricks_cluster` to allow
users to disable usage in certain contexts
([#3972](#3972)).
* Fix `databricks_sql_table` treatment of properties
([#3925](#3925)).
* Force send fields for settings resources
([#3978](#3978)).
* Handle cluster deletion in `databricks_library` read
([#3909](#3909)).
* Make subscriptions optional for SqlAlertTask
([#3983](#3983)).
* Permanently delete `ERROR` and `TERMINATED` state clusters if their
creation fails
([#4021](#4021)).


### Documentation

* Add troubleshooting guide for Provider
"registry.terraform.io/databricks/databricks" planned an invalid value
([#3961](#3961)).
* Adopt official naming of Mosaic AI Vector Search
([#3971](#3971)).
* Document Terraform 1.0 as minimum version
([#3952](#3952)).
* Mention Salesforce as supported type in `databricks_connection`
([#3949](#3949)).
* Reimplement Azure Databricks deployment guide to use VNet injection &
NPIP
([#3986](#3986)).
* Resolves
[#3127](#3127):
Remove deprecated account_id field from mws_credentials resource
([#3974](#3974)).
* Small Grammar Corrections in Docs
([#4006](#4006)).
* Update `databricks_vector_search_index` docs to match latest SDK
([#4008](#4008)).
* Update aws_unity_catalog_assume_role_policy.md
([#3968](#3968)).
* Update documentation regarding authentication with Azure-managed
Service Principal using GITHUB OIDC
([#3932](#3932)).
* Update metastore_assignment.md to properly reflect possible usage
([#3967](#3967)).
* Update minimum supported terraform version to 1.1.5
([#3965](#3965)).
* Update resources diagram to include newer resources
([#3962](#3962)).
* Update workspace_binding import command
([#3944](#3944)).
* fix possible values for `securable_type` in
`databricks_workspace_binding`
([#3942](#3942)).


### Internal Changes

* Add `AddPlanModifer` method for AttributeBuilder
([#4009](#4009)).
* Add integration tests for volumes and quality monitor plugin framework
([#3975](#3975)).
* Add support for `computed` tag in TfSDK Structs
([#4005](#4005)).
* Added `databricks_quality_monitor` resource and `databricks_volumes`
data source to plugin framework
([#3958](#3958)).
* Allow vector search tests to fail
([#3959](#3959)).
* Clean up comments in library resource
([#4015](#4015)).
* Fix irregularities in plugin framework converter function errors
([#4010](#4010)).
* Make test utils public and move integration test for quality monitor
([#3993](#3993)).
* Migrate Share resource to Go SDK
([#3916](#3916)).
* Migrate `databricks_cluster` data source to plugin framework
([#3988](#3988)).
* Migrate imports for terraform plugin framework + update init test
provider factory
([#3943](#3943)).
* Move volumes test next to plugin framework data source
([#3995](#3995)).
* Refactor provider and related packages
([#3940](#3940)).
* Support import in acceptance test + adding import state for quality
monitor
([#3994](#3994)).
* Library plugin framework migration
([#3979](#3979)).
* Fix `TestAccClusterResource_WorkloadType`
([#3989](#3989)).


### Dependency Updates

* Bump github.com/hashicorp/hcl/v2 from 2.21.0 to 2.22.0
([#3948](#3948)).
* Update Go SDK to 0.46.0
([#4007](#4007)).


### Exporter

* Don't generate instance pools if the pool name is empty
([#3960](#3960)).
* Expand list of non-interactive clusters
([#4023](#4023)).
* Ignore databricks_artifact_allowlist with zero artifact_matcher blocks
([#4019](#4019)).


## [Release] Release v1.51.0

### Breaking Changes

With this release, only protocol version 6 will be supported which is
compatible with terraform CLI version 1.1.5 and later. If you are using
an older version of the terraform CLI, please upgrade it to use this and
further releases of Databricks terraform provider.

### New Features and Improvements

* Automatically create `parent_path` folder when creating
`databricks_dashboard resource` if it doesn't exist
([#3778](#3778)).


### Bug Fixes

* Fixed logging for underlying Go SDK
([#3917](#3917)).
* Remove not necessary field in `databricks_job` schema
([#3907](#3907)).


### Internal Changes

* Add AttributeBuilder for Plugin Framework schema
([#3922](#3922)).
* Add CustomizableSchema for Plugin Framework
([#3927](#3927)).
* Add StructToSchema for Plugin Framework
([#3928](#3928)).
* Add codegen template and generated files for tfsdk structs
([#3911](#3911)).
* Add converter functions and tests for plugin framework
([#3914](#3914)).
* Added support to use protocol version 6 provider server for SDK plugin
([#3862](#3862)).
* Bump Go SDK to v0.45.0
([#3933](#3933)).
* Change name with the aliases in codegen template
([#3936](#3936)).
* Update jd version from latest to 1.8.1
([#3915](#3915)).
* Upgrade `staticcheck` to v0.5.1 to get Go 1.23 support
([#3931](#3931)).
* OPENAPI_SHA check
([#3935](#3935)).
* Use generic error for missing clusters
([#3938](#3938))


### Exporter

* Better support for notebooks with /Workspace path
([#3901](#3901)).
* Improve exporting of DLT and test coverage
([#3898](#3898)).
@alexott alexott mentioned this pull request Nov 13, 2024
Closed
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanmay-db tanmay-db tanmay-db approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@rauchy @tanmay-db