Skip to content

Commit

Permalink
Added documentation for job multiple task orchestration
Browse files Browse the repository at this point in the history
  • Loading branch information
nfx committed Oct 12, 2021
1 parent 3d3fde8 commit 08abb9b
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## 0.3.9

* Added initial support for multiple task orchestration in `databricks_job` [#853](https://github.com/databrickslabs/terraform-provider-databricks/pull/853)
* Fixed provider crash for new terraform states related to bug [#813](https://github.com/hashicorp/terraform-plugin-sdk/issues/813) in Terraform SDK v2.8.0 ([#854](https://github.com/databrickslabs/terraform-provider-databricks/issues/854))

Updated dependency versions:
Expand Down
42 changes: 42 additions & 0 deletions docs/resources/job.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,48 @@ output "job_url" {
}
```

## Jobs with Multiple Tasks

-> **Note** In terraform configuration, you must define tasks in alphabetical order of their `task_key` arguments, so that you get consistent and readable diff. Whenever tasks are added or removed, or `task_key` is renamed, you'll observe a change in the majority of tasks. It's related to the fact that the current version of the provider treats `task` blocks as an ordered list. Alternatively, `task` block could have been an unordered set, though end-users would see the entire block replaced upon a change in single property of the task.

It is possible to create jobs with multiple tasks using the `task` blocks:

```hcl
resource "databricks_job" "this" {
name = "Job with multiple tasks"
task {
task_key = "a"
new_cluster {
num_workers = 1
spark_version = data.databricks_spark_version.latest.id
node_type_id = data.databricks_node_type.smallest.id
}
notebook_task {
notebook_path = databricks_notebook.this.path
}
}
task {
task_key = "b"
depends_on {
task_key = "a"
}
existing_cluster_id = databricks_cluster.shared.id
spark_jar_task {
main_class_name = "com.acme.data.Main"
}
}
}
```

Every `task` block can have almos all available arguments with the addition of `task_key` attribute and `depends_on` blocks to define cross-task dependencies.

## Argument Reference

The following arguments are required:
Expand Down

0 comments on commit 08abb9b

Please sign in to comment.