[ISSUE] databricks_permissions fails with Provider produced inconsistent result when provided upgraded to 0.3.0

[nirav24]

I was running below code with Databricks provider 0.2.4 and it was working fine. I am currently looking to upgrade provider to latest version 0.3.0 and terraform apply is failing with below error.

Terraform Version

Run terraform -v to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.

terraform -v
Terraform v0.12.21

Affected Resource(s)

Please list the resources as a list, for example:

• databricks_permissions

Environment variable names

To get relevant environment variable names please copypaste the output of the following command:
$ env | sort | grep -E 'DATABRICKS|AWS|AZURE|ARM|TEST' | awk -F= '{print $1}'

ARM_ACCESS_KEY
ARM_CLIENT_ID
ARM_CLIENT_SECRET
ARM_SUBSCRIPTION_ID
ARM_TENANT_ID

Terraform Configuration Files

resource "databricks_cluster_policy" "vm_policy" {
  name       = "db_vm_cluster_policy"
  definition = jsonencode(merge(local.default_policy))
}

resource "databricks_permissions" "can_use_cluster_policy_vm_type" {
  cluster_policy_id = databricks_cluster_policy.vm_policy.id

  access_control {
    group_name = "admins"
    permission_level = "CAN_USE"
  }
}

Debug Output

TF_LOG=DEBUG terraform plan 2>&1 | grep databricks | sed -E 's/^.* plugin[^:]+: (.*)$/\1/'

databricks_permissions.can_use_cluster_policy_vm_type: Creating...
2021/02/04 15:45:00 [DEBUG] databricks_permissions.can_use_cluster_policy_vm_type: applying the planned Create change
Using Azure Service Principal client secret authentication: timestamp=2021-02-04T15:45:00.156-0500
Getting Workspace ID via management token.: timestamp=2021-02-04T15:45:00.157-0500
GET https://management.azure.com/subscriptions/xxxx-xxx-xxxxxx-xxxxx/resourceGroups/xxxx-xxx-xxx/providers/Microsoft.Databricks/workspaces/xxxx-xxx-xxx-adb-xxx?api-version=2018-04-01: timestamp=2021-02-04T15:45:00.708-0500
200 OK {
    "managedResourceGroupId": "/subscriptions/xxxx-xxx-xxxxxx-xxxxx/resourceGroups/databricks-rg-xxxx-xxx-xxx",
        "value": "databricks-private"
        "value": "databricks-public"
Creating workspace token: timestamp=2021-02-04T15:45:01.450-0500
Setting 'X-Databricks-Azure-SP-Management-Token' header: timestamp=2021-02-04T15:45:01.450-0500
POST https://adb-XXXXXXXX.azuredatabricks.net/api/2.0/token/create {
200 OK {
} <- POST https://adb-XXXXXXXX.azuredatabricks.net/api/2.0/token/create: timestamp=2021-02-04T15:45:02.939-0500
PUT /preview/permissions/cluster-policies/XXXXXXXX {
200 OK {
GET /preview/permissions/cluster-policies/XXXXXXXX: timestamp=2021-02-04T15:45:03.427-0500
200 OK {
GET /preview/scim/v2/Me: timestamp=2021-02-04T15:45:03.729-0500
200 OK {
2021/02/04 15:45:04 [DEBUG] databricks_permissions.can_use_cluster_policy_vm_type: apply errored, 
            but we're indicating that via the Error pointer rather than returning it: 
            Provider produced inconsistent result after apply: When applying changes to 
            databricks_permissions.can_use_cluster_policy_vm_type, 
            provider "registry.terraform.io/-/databricks" produced an unexpected new value for was present, but now absent.
2021/02/04 15:45:04 [ERROR] <root>: eval: *terraform.EvalApplyPost, err: Provider produced inconsistent result after apply: 
            When applying changes to databricks_permissions.can_use_cluster_policy_vm_type, provider "registry.terraform.io/-/databricks" 
            produced an unexpected new value for was present, but now absent.
2021/02/04 15:45:04 [ERROR] <root>: eval: *terraform.EvalSequence, err: Provider produced inconsistent result after apply: 
            When applying changes to databricks_permissions.can_use_cluster_policy_vm_type, provider "registry.terraform.io/-/databricks" 
            produced an unexpected new value for was present, but now absent.
databricks_permissions.can_use_cluster_policy_vm_type, provider
"registry.terraform.io/-/databricks" produced an unexpected new value for was
path=/Users/npatel/.terraform.d/plugins/darwin_amd64/terraform-provider-databricks_v0.3.0 pid=50166

Expected Behavior

I think, it should be able to assign policy to admins group. (I am not sure if Databricks doesn't allow to update admins group)

Actual Behavior

databricks_permissions.can_use_cluster_policy_vm_type: Creating...

Error: Provider produced inconsistent result after apply

When applying changes to
databricks_permissions.can_use_cluster_policy_vm_type, provider
"registry.terraform.io/-/databricks" produced an unexpected new value for was
present, but now absent.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

Are there anything atypical about your accounts that we should know?

[nfx]

@nirav24 https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs/resources/permissions

This resource allows you to generically manage permissions for other resources in Databricks workspace. It would guarantee, that only admins, authenticated principal and those declared within access_control blocks would have specified access. It is not possible to remove management rights from admins group.

i'll add validation to prevent admins as a value for group_name. just because admins will always have permission on everything.

[nirav24]

@nirav24 https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs/resources/permissions

This resource allows you to generically manage permissions for other resources in Databricks workspace. It would guarantee, that only admins, authenticated principal and those declared within access_control blocks would have specified access. It is not possible to remove management rights from admins group.

i'll add validation to prevent admins as a value for group_name. just because admins will always have permission on everything.

Thanks @nfx for quick reply. I am trying to apply policies which will restrict VM types can be used even for admins. Do you think, that policy will be surpassed as well?

[nfx]

@nirav24 it's not possible to restrict anything for admins :) just remove people from admins group.

[nirav24]

i'll add validation to prevent admins as a value for group_name. just because admins will always have permission on everything.

I can raise a PR if you wish

[nfx]

@nirav24 i would appreciate, review and merge it! :)