Skip to content

Commit

Permalink
Added token block to databricks_mws_workspaces (#969)
Browse files Browse the repository at this point in the history
  • Loading branch information
@nfx
nfx committed Dec 15, 2021
1 parent 8a1921a commit ed62099
Show file tree
Hide file tree
Showing 9 changed files with 636 additions and 186 deletions.
11 changes: 10 additions & 1 deletion .vscode/testing.code-snippets
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,14 @@
"}"
],
"description": "Method with single error result"
}
},
"if err != nil return err": {
"prefix": "ife",
"body": [
"if err != nil {",
" return err",
"}"
],
"description": "if err != nil return err"
}
}
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
* Fixed refresh of `library` blocks on a stopped `databricks_cluster` ([#952](https://github.com/databrickslabs/terraform-provider-databricks/issues/952)).
* Added `databricks_clusters` data resource to list all clusters in the workspace ([#955](https://github.com/databrickslabs/terraform-provider-databricks/pull/955)).
* Whenever a library fails to get installed on a running `databricks_cluster`, we now automatically remove this library, so that the clean state of managed libraries is properly maintained. Without this fix users had to manually go to Clusters UI and remove library from a cluster, where it failed to install. Libraries add up to CREATE and UPDATE timeouts of `databricks_cluster` resource. ([#599](https://github.com/databrickslabs/terraform-provider-databricks/issues/599)).
* Added `token` block to `databricks_mws_workspaces` to avoid unnecessary provider aliasing ([#957](https://github.com/databrickslabs/terraform-provider-databricks/issues/957)).
* Added new experimental resources and increased test coverage.

## 0.4.0
Expand Down
21 changes: 5 additions & 16 deletions docs/guides/aws-workspace.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -255,29 +255,18 @@ resource "databricks_mws_workspaces" "this" {
credentials_id = databricks_mws_credentials.this.credentials_id
storage_configuration_id = databricks_mws_storage_configurations.this.storage_configuration_id
network_id = databricks_mws_networks.this.network_id
token {
comment = "Terraform"
}
}
output "databricks_host" {
value = databricks_mws_workspaces.this.workspace_url
}
// initialize provider in normal mode
provider "databricks" {
// in normal scenario you won't have to give providers aliases
alias = "created_workspace"
host = databricks_mws_workspaces.this.workspace_url
}
// create PAT token to provision entities within workspace
resource "databricks_token" "pat" {
provider = databricks.created_workspace
comment = "Terraform Provisioning"
lifetime_seconds = 86400
}
// export token for integration tests to run on
output "databricks_token" {
value = databricks_token.pat.token_value
value = databricks_mws_workspaces.this.token[0].token_value
sensitive = true
}
```
Expand Down
42 changes: 30 additions & 12 deletions docs/resources/mws_workspaces.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ subcategory: "AWS"

This resource allows you to set up [workspaces in E2 architecture on AWS](https://docs.databricks.com/getting-started/overview.html#e2-architecture-1). Please follow this [complete runnable example](../guides/aws-workspace.md) with new VPC and new workspace setup.

-> **Note** Initialize provider with `alias = "mws"` and use `provider = databricks.mws` for all `databricks_mws_*` resources. We require all `databricks_mws_*` resources to be created within its own dedicated terraform module of your environment. Usually this module creates VPC and IAM roles as well. Code that creates workspaces and code that [manages workspaces](../guides/workspace-management.md) must be in separate terraform modules to avoid common confusion between `provider = databricks.mws` and `provider = databricks.created_workspace`. This is why we specify `databricks_host` and `databricks_token` outputs, that have to be used in the latter modules:

```hcl
provider "databricks" {
host = module.ai.databricks_host
token = module.ai.databricks_token
}
```

## Example Usage

![Simplest multiworkspace](https://github.com/databrickslabs/terraform-provider-databricks/raw/master/docs/simplest-multiworkspace.png)
Expand Down Expand Up @@ -65,21 +74,13 @@ resource "databricks_mws_workspaces" "this" {
credentials_id = databricks_mws_credentials.this.credentials_id
storage_configuration_id = databricks_mws_storage_configurations.this.storage_configuration_id
network_id = databricks_mws_networks.this.network_id
}
provider "databricks" {
// in normal scenario you won't have to give providers aliases
alias = "created_workspace"
host = databricks_mws_workspaces.this.workspace_url
token {}
}
// create PAT token to provision entities within workspace
resource "databricks_token" "pat" {
provider = databricks.created_workspace
comment = "Terraform Provisioning"
// 1 day token
lifetime_seconds = 86400
output "databricks_token" {
value = databricks_mws_workspaces.this.token[0].token_value
sensitive = true
}
```

Expand Down Expand Up @@ -170,6 +171,13 @@ resource "databricks_mws_workspaces" "this" {
credentials_id = databricks_mws_credentials.this.credentials_id
storage_configuration_id = databricks_mws_storage_configurations.this.storage_configuration_id
token {}
}
output "databricks_token" {
value = databricks_mws_workspaces.this.token[0].token_value
sensitive = true
}
```

Expand All @@ -189,6 +197,16 @@ The following arguments are available and cannot be changed after workspace is c
* `storage_configuration_id` - `storage_configuration_id` from [storage configuration](mws_storage_configurations.md)
* `private_access_settings_id` - (Optional) Canonical unique identifier of [databricks_mws_private_access_settings](mws_private_access_settings.md) in Databricks Account

## token block

You can specify a `token` block in the body of workspace resource, so that Terraform manages the refresh of PAT token for the deployment user. The other option is to create [databricks_obo_token](obo_token.md), though it requires Premium or Enterprise plan enabled as well as more complex setup. Token block exposes `token_value`, that holds sensitive PAT token and optionally it can accept two arugments:

-> **Note** Tokens managed by `token {}` block are recreated when expired.


* `comment` - (Optional) Comment, that will appear in "User Settings / Access Tokens" page on Workspace UI. By default it's "Terraform PAT".
* `lifetime_seconds` - (Optional) Token expiry lifetime. By default its 2592000 (30 days).

The following arguments could be modified after the workspace is running:

* `network_id` - (Optional) `network_id` from [networks](mws_networks.md). Modifying [networks on running workspaces](mws_networks.md#modifying-networks-on-running-workspaces) would require three separate `terraform apply` steps.
Expand Down
96 changes: 0 additions & 96 deletions mws/mws.go
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
package mws

import "encoding/json"

// StsRole is the object that contains cross account role arn and external app id
type StsRole struct {
RoleArn string `json:"role_arn,omitempty"`
Expand Down Expand Up @@ -63,94 +61,6 @@ type Network struct {
CreationTime int64 `json:"creation_time,omitempty" tf:"computed"`
}

// List of workspace statuses for provisioning the workspace
const (
WorkspaceStatusNotProvisioned = "NOT_PROVISIONED"
WorkspaceStatusProvisioning = "PROVISIONING"
WorkspaceStatusRunning = "RUNNING"
WorkspaceStatusFailed = "FAILED"
WorkspaceStatusCanceled = "CANCELLED"
)

// WorkspaceStatusesNonRunnable is a list of statuses in which the workspace is not runnable
var WorkspaceStatusesNonRunnable = []string{WorkspaceStatusCanceled, WorkspaceStatusFailed}

type GCP struct {
ProjectID string `json:"project_id"`
}

type CloudResourceBucket struct {
GCP *GCP `json:"gcp"`
}

type GCPManagedNetworkConfig struct {
SubnetCIDR string `json:"subnet_cidr"`
GKEClusterPodIPRange string `json:"gke_cluster_pod_ip_range"`
GKEClusterServiceIPRange string `json:"gke_cluster_service_ip_range"`
}

type GCPCommonNetworkConfig struct {
GKEConnectivityType string `json:"gke_connectivity_type"`
GKEClusterMasterIPRange string `json:"gke_cluster_master_ip_range"`
}

type GCPNetwork struct {
GCPManagedNetworkConfig *GCPManagedNetworkConfig `json:"gcp_managed_network_config"`
GCPCommonNetworkConfig *GCPCommonNetworkConfig `json:"gcp_common_network_config"`
}

// Workspace is the object that contains all the information for deploying a workspace
type Workspace struct {
AccountID string `json:"account_id"`
WorkspaceName string `json:"workspace_name"`
DeploymentName string `json:"deployment_name,omitempty"`
AwsRegion string `json:"aws_region,omitempty"` // required for AWS, not allowed for GCP
CredentialsID string `json:"credentials_id,omitempty"` // required for AWS, not allowed for GCP
CustomerManagedKeyID string `json:"customer_managed_key_id,omitempty"` // just for compatibility, will be removed
StorageConfigurationID string `json:"storage_configuration_id,omitempty"` // required for AWS, not allowed for GCP
ManagedServicesCustomerManagedKeyID string `json:"managed_services_customer_managed_key_id,omitempty"`
StorageCustomerManagedKeyID string `json:"storage_customer_managed_key_id,omitempty"`
PricingTier string `json:"pricing_tier,omitempty" tf:"computed"`
PrivateAccessSettingsID string `json:"private_access_settings_id,omitempty"`
NetworkID string `json:"network_id,omitempty"`
IsNoPublicIPEnabled bool `json:"is_no_public_ip_enabled"`
WorkspaceID int64 `json:"workspace_id,omitempty" tf:"computed"`
WorkspaceURL string `json:"workspace_url,omitempty" tf:"computed"`
WorkspaceStatus string `json:"workspace_status,omitempty" tf:"computed"`
WorkspaceStatusMessage string `json:"workspace_status_message,omitempty" tf:"computed"`
CreationTime int64 `json:"creation_time,omitempty" tf:"computed"`

ExternalCustomerInfo *externalCustomerInfo `json:"external_customer_info,omitempty"`

CloudResourceBucket *CloudResourceBucket `json:"cloud_resource_bucket,omitempty"`
Network *GCPNetwork `json:"network,omitempty"`
Cloud string `json:"cloud,omitempty" tf:"computed"`
Location string `json:"location,omitempty"`
}

// this type alias hack is required for Marshaller to work without an infinite loop
type aWorkspace Workspace

// MarshalJSON is required to overcome the limitations of `omitempty` usage with reflect_resource.go
// for workspace creation in Accounts API for AWS and GCP. It exits early on AWS and picks only
// the relevant fields for GCP.
func (w *Workspace) MarshalJSON() ([]byte, error) {
if w.Cloud != "gcp" {
return json.Marshal(aWorkspace(*w))
}
workspaceCreationRequest := map[string]interface{}{
"account_id": w.AccountID,
"cloud": w.Cloud,
"cloud_resource_bucket": w.CloudResourceBucket,
"location": w.Location,
"workspace_name": w.WorkspaceName,
}
if w.Network != nil {
workspaceCreationRequest["network"] = w.Network
}
return json.Marshal(workspaceCreationRequest)
}

// VPCEndpoint is the object that contains all the information for registering an VPC endpoint
type VPCEndpoint struct {
VPCEndpointID string `json:"vpc_endpoint_id,omitempty" tf:"computed"`
Expand All @@ -175,9 +85,3 @@ type PrivateAccessSettings struct {
PrivateAccessLevel string `json:"private_access_level,omitempty" tf:"default:ANY"`
AllowedVpcEndpointIDS []string `json:"allowed_vpc_endpoint_ids,omitempty"`
}

type externalCustomerInfo struct {
CustomerName string `json:"customer_name"`
AuthoritativeUserEmail string `json:"authoritative_user_email"`
AuthoritativeUserFullName string `json:"authoritative_user_full_name"`
}
Loading

0 comments on commit ed62099

Please sign in to comment.