Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove managed library that failed to install on refresh #599

Closed
nfx opened this issue Apr 19, 2021 · 6 comments · Fixed by #956
Closed

Remove managed library that failed to install on refresh #599

nfx opened this issue Apr 19, 2021 · 6 comments · Fixed by #956
Assignees
@nfx
Labels
suppress diff issue related to configuration drift, most likely from Cluster Manager
Milestone
v0.4.1

Comments

@nfx
Copy link
Contributor

nfx commented Apr 19, 2021

Remove managed library that failed to install on refresh
@nfx nfx added this to the v0.3.4 milestone Apr 22, 2021
@nfx nfx modified the milestones: v0.3.4, v0.3.5 May 11, 2021
@nfx
Copy link
Contributor Author

nfx commented May 12, 2021

how to reproduce:

  1. install the library that failed to install on a cluster
  2. see TF apply end with the error
  3. run apply again
  4. see same installation error :(

@nfx nfx modified the milestones: v0.3.5, v0.3.6 Jun 2, 2021
@nfx nfx mentioned this issue Jul 4, 2021
Closed
@nfx nfx removed this from the v0.3.6 milestone Jul 8, 2021
@nfx
Copy link
Contributor Author

nfx commented Jul 8, 2021

@psg2 this might be interesting for you to look at

@psg2
Copy link
Contributor

psg2 commented Jul 9, 2021

@psg2 this might be interesting for you to look at

I will give a look into this next week.

Unrelated to the issue itself, but just found it interesting that the terraform refresh command is being deprecated. Ref: https://www.terraform.io/docs/cli/commands/refresh.html

@nfx
Copy link
Contributor Author

nfx commented Jul 12, 2021

@psg2 Refresh is more of a stage name, that calls Read callback of a resource

@nfx
Copy link
Contributor Author

nfx commented Jul 21, 2021

@psg2 do you still want to pick this one up?

@psg2
Copy link
Contributor

psg2 commented Jul 21, 2021

I won't have time to do it soon so I won't make a compromise for now. ):

@nfx nfx mentioned this issue Sep 3, 2021
Closed
@nfx nfx added this to the v0.4.0 milestone Nov 10, 2021
@nfx nfx mentioned this issue Nov 10, 2021
Closed
@nfx nfx added the suppress diff issue related to configuration drift, most likely from Cluster Manager label Dec 2, 2021
@nfx nfx modified the milestones: v0.4.0, v0.4.1 Dec 9, 2021
@nfx nfx self-assigned this Dec 11, 2021
nfx added a commit that referenced this issue Dec 11, 2021
@nfx
Remove failed libraries on running clusters
bea055a 
Whenever a library fails to get installed on a running cluster, we automatically remove it, so that the clean state of managed libraries is properly maintained. Without this fix users had to manually go to Clusters UI and remove library from a cluster, where it failed to install. Libraries add up to CREATE and UPDATE timeouts of `databricks_cluster` resource.

Fixes #599
@nfx nfx mentioned this issue Dec 11, 2021
Merged
@nfx nfx closed this as completed in #956 Dec 12, 2021
nfx added a commit that referenced this issue Dec 16, 2021
@nfx
Release v0.4.1
b9c8025 
* Added `databricks_library` resource to install library on `databricks_cluster` ([#904](#904)).
* Added `databricks_clusters` data resource to list all clusters in the workspace, which might be used to install `databricks_library` on all clusters ([#955](#955)).
* Fixed refresh of `library` blocks on a stopped `databricks_cluster` ([#952](#952)).
* Whenever a library fails to get installed on a running `databricks_cluster`, we now automatically remove this library, so that the clean state of managed libraries is properly maintained. Without this fix users had to manually go to Clusters UI and remove library from a cluster, where it failed to install. Libraries add up to CREATE and UPDATE timeouts of `databricks_cluster` resource. ([#599](#599)).
* Added `token` block to `databricks_mws_workspaces` to avoid unnecessary provider aliasing ([#957](#957)).
* Fixed disabling `databricks_global_init_script` ([#958](#958)).
* Fixed configuration drift issues with `aws_attributes`, `azure_attributes`, `gcp_attributes`, and `email_notifications` configuration blocks in `databricks_cluster`, `databricks_job`, and `databricks_instance_pool` resources ([#981](#981)).
* Improved Databricks CLI auth by eagerly resolving `host`, `username`, `password`, and `token` from the specified `profile`. Added explicit logging of auth parameters in debug logs ([#965](#965)).
* TLS timeouts, which may occur during Azure MSI auth, are no longer failing API requests and retried within a normal policy ([#966](#966)).
* `debug_headers` provider conf is also logging the `Host` header to help troubleshooting auth issues ([#964](#964)).
* Added new experimental resources and increased test coverage.

Updated dependency versions:

* Bump github.com/golang-jwt/jwt/v4 from 4.1.0 to 4.2.0
* Bump google.golang.org/api from 0.60.0 to 0.63.0
* Bump github.com/Azure/go-autorest/autorest from 0.11.22 to 0.11.23
* Bump github.com/Azure/go-autorest/autorest/azure/auth from 0.5.9 to 0.5.10
* Bump gopkg.in/ini.v1 from 1.66.0 to 1.66.2
* Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.9.0 to 2.10.0
@nfx nfx mentioned this issue Dec 16, 2021
Merged
nfx added a commit that referenced this issue Dec 16, 2021
@nfx
Release v0.4.1 (#983)
da93abe 
Change log:

* Added `databricks_library` resource to install library on `databricks_cluster` ([#904](#904)).
* Added `databricks_clusters` data resource to list all clusters in the workspace, which might be used to install `databricks_library` on all clusters ([#955](#955)).
* Fixed refresh of `library` blocks on a stopped `databricks_cluster` ([#952](#952)).
* Whenever a library fails to get installed on a running `databricks_cluster`, we now automatically remove this library, so that the clean state of managed libraries is properly maintained. Without this fix users had to manually go to Clusters UI and remove library from a cluster, where it failed to install. Libraries add up to CREATE and UPDATE timeouts of `databricks_cluster` resource. ([#599](#599)).
* Added `token` block to `databricks_mws_workspaces` to avoid unnecessary provider aliasing ([#957](#957)).
* Fixed disabling `databricks_global_init_script` ([#958](#958)).
* Fixed configuration drift issues with `aws_attributes`, `azure_attributes`, `gcp_attributes`, and `email_notifications` configuration blocks in `databricks_cluster`, `databricks_job`, and `databricks_instance_pool` resources ([#981](#981)).
* Improved Databricks CLI auth by eagerly resolving `host`, `username`, `password`, and `token` from the specified `profile`. Added explicit logging of auth parameters in debug logs ([#965](#965)).
* TLS timeouts, which may occur during Azure MSI auth, are no longer failing API requests and retried within a normal policy ([#966](#966)).
* `debug_headers` provider conf is also logging the `Host` header to help troubleshooting auth issues ([#964](#964)).
* Added new experimental resources and increased test coverage.

Updated dependency versions:

* Bump github.com/golang-jwt/jwt/v4 from 4.1.0 to 4.2.0
* Bump google.golang.org/api from 0.60.0 to 0.63.0
* Bump github.com/Azure/go-autorest/autorest from 0.11.22 to 0.11.23
* Bump github.com/Azure/go-autorest/autorest/azure/auth from 0.5.9 to 0.5.10
* Bump gopkg.in/ini.v1 from 1.66.0 to 1.66.2
* Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.9.0 to 2.10.0
michael-berk pushed a commit to michael-berk/terraform-provider-databricks that referenced this issue Feb 15, 2023
@nfx
Release v0.4.1 (databricks#983)
adf4a40 
Change log:

* Added `databricks_library` resource to install library on `databricks_cluster` ([databricks#904](databricks#904)).
* Added `databricks_clusters` data resource to list all clusters in the workspace, which might be used to install `databricks_library` on all clusters ([databricks#955](databricks#955)).
* Fixed refresh of `library` blocks on a stopped `databricks_cluster` ([databricks#952](databricks#952)).
* Whenever a library fails to get installed on a running `databricks_cluster`, we now automatically remove this library, so that the clean state of managed libraries is properly maintained. Without this fix users had to manually go to Clusters UI and remove library from a cluster, where it failed to install. Libraries add up to CREATE and UPDATE timeouts of `databricks_cluster` resource. ([databricks#599](databricks#599)).
* Added `token` block to `databricks_mws_workspaces` to avoid unnecessary provider aliasing ([databricks#957](databricks#957)).
* Fixed disabling `databricks_global_init_script` ([databricks#958](databricks#958)).
* Fixed configuration drift issues with `aws_attributes`, `azure_attributes`, `gcp_attributes`, and `email_notifications` configuration blocks in `databricks_cluster`, `databricks_job`, and `databricks_instance_pool` resources ([databricks#981](databricks#981)).
* Improved Databricks CLI auth by eagerly resolving `host`, `username`, `password`, and `token` from the specified `profile`. Added explicit logging of auth parameters in debug logs ([databricks#965](databricks#965)).
* TLS timeouts, which may occur during Azure MSI auth, are no longer failing API requests and retried within a normal policy ([databricks#966](databricks#966)).
* `debug_headers` provider conf is also logging the `Host` header to help troubleshooting auth issues ([databricks#964](databricks#964)).
* Added new experimental resources and increased test coverage.

Updated dependency versions:

* Bump github.com/golang-jwt/jwt/v4 from 4.1.0 to 4.2.0
* Bump google.golang.org/api from 0.60.0 to 0.63.0
* Bump github.com/Azure/go-autorest/autorest from 0.11.22 to 0.11.23
* Bump github.com/Azure/go-autorest/autorest/azure/auth from 0.5.9 to 0.5.10
* Bump gopkg.in/ini.v1 from 1.66.0 to 1.66.2
* Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.9.0 to 2.10.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nfx nfx

Labels
suppress diff issue related to configuration drift, most likely from Cluster Manager
Projects
None yet
Milestone
v0.4.1
Development

Successfully merging a pull request may close this issue.

Remove failed libraries on running clusters databricks/terraform-provider-databricks
2 participants
@nfx @psg2