Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend DbtTask with additional supported fields and document them #1537

Merged
merged 5 commits into from
Aug 30, 2022

Conversation

gaborratky-db
Copy link
Contributor

What changes are proposed in this pull request?

  • Add additional fields that the upcoming dbt task type [1] supports:
    • ProfilesDirectory - Optional field to configure where dbt should look for profiles.yml within the repository.
    • WarehouseId - Optional field configuring a Databricks SQL Warehouse that dbt should run the queries against.
  • Allow configuring a job with a single dbt task
  • Document dbt_task in docs/resources/job.md.

How is this tested?

E2E tested with https://github.com/gaborratky-db/jaffle_shop/tree/main/terrafood

[1] https://databricks.com/blog/2022/06/29/top-5-workflows-announcements-at-data-ai-summit.html

Copy link
Contributor

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

docs/resources/job.md Outdated Show resolved Hide resolved
docs/resources/job.md Outdated Show resolved Hide resolved
Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com>
@gaborratky-db gaborratky-db marked this pull request as draft August 16, 2022 15:32
@nfx
Copy link
Contributor

nfx commented Aug 29, 2022

@gaborratky-db is this functionality working now? :)

@gaborratky-db
Copy link
Contributor Author

@gaborratky-db is this functionality working now? :)

@nfx, yes, dbt tasks are available for all Premium and Enterprise Databricks customers.

@gaborratky-db gaborratky-db marked this pull request as ready for review August 30, 2022 20:19
@codecov-commenter
Copy link

Codecov Report

Merging #1537 (ba91934) into master (a0af466) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1537   +/-   ##
=======================================
  Coverage   90.15%   90.15%           
=======================================
  Files         132      132           
  Lines       10621    10621           
=======================================
  Hits         9575     9575           
  Misses        667      667           
  Partials      379      379           
Impacted Files Coverage Δ
jobs/resource_job.go 95.19% <ø> (ø)

@nfx nfx merged commit a016f86 into databricks:master Aug 30, 2022
@gaborratky-db gaborratky-db deleted the extend-jobs-dbt-task branch August 30, 2022 20:54
@nfx nfx mentioned this pull request Sep 12, 2022
michael-berk pushed a commit to michael-berk/terraform-provider-databricks that referenced this pull request Feb 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants