[FEATURE] Support for shared job clusters in multi-task jobs #1082
Labels
feature
New feature or request
Comments
nfx
changed the title
|
@cbcoutinho this feature is not yet implemented in provider |
nfx
added a commit
that referenced
this issue
Feb 11, 2022
# Version changelog ## 0.4.9 * Prevent creation of `databricks_group` with `users` and `admins` reserved names ([#1089](#1089)). * Added support for shared clusters in multi-task `databricks_job` ([#1082](#1082)). * Added diff suppression for `external_id` in `databricks_group` ([#1099](#1099)). * Added diff suppression for `external_id` in `databricks_user` ([#1097](#1097)). * Added `users`, `service_principals`, and `child_groups` exported properties to `databricks_group` data resource ([#1085](#1085)). * Added various documentation improvements.
Merged
nfx
added a commit
that referenced
this issue
Feb 11, 2022
* Release v0.4.9 # Version changelog ## 0.4.9 * Prevent creation of `databricks_group` with `users` and `admins` reserved names ([#1089](#1089)). * Added support for shared clusters in multi-task `databricks_job` ([#1082](#1082)). * Added diff suppression for `external_id` in `databricks_group` ([#1099](#1099)). * Added diff suppression for `external_id` in `databricks_user` ([#1097](#1097)). * Added `users`, `service_principals`, and `child_groups` exported properties to `databricks_group` data resource ([#1085](#1085)). * Added various documentation improvements.
michael-berk
pushed a commit
to michael-berk/terraform-provider-databricks
that referenced
this issue
Feb 15, 2023
* Release v0.4.9 # Version changelog ## 0.4.9 * Prevent creation of `databricks_group` with `users` and `admins` reserved names ([databricks#1089](databricks#1089)). * Added support for shared clusters in multi-task `databricks_job` ([databricks#1082](databricks#1082)). * Added diff suppression for `external_id` in `databricks_group` ([databricks#1099](databricks#1099)). * Added diff suppression for `external_id` in `databricks_user` ([databricks#1097](databricks#1097)). * Added `users`, `service_principals`, and `child_groups` exported properties to `databricks_group` data resource ([databricks#1085](databricks#1085)). * Added various documentation improvements.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Affected Resource(s)
Databricks jobs: https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs/resources/job
Expected Details
The documentation only mentions using interactive clusters for shared clusters; however those are not the same as job clusters. Via the Databricks UI it's possible to select a single cluster for multiple tasks, but the provider and/or documentation do not distinguish between the two
List of things to potentially add/remove:
This is a list of things to manipulate in the docs:
Important Factoids
This is the most recent blog post on the Job API updates that mention shared job clusters. Looking at the publish date, this may have been just released. That may explain why it's not mentioned in the docs.
https://docs.databricks.com/data-engineering/jobs/jobs-api-updates.html
References
Databricks TF provider docs mentioning
existing_cluster_id: https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs/resources/jobDatabricks docs regarding shared job clusters: https://docs.microsoft.com/en-us/azure/databricks/jobs#use-shared-job-clusters
The text was updated successfully, but these errors were encountered: