Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use print for outputting codegen #86

Merged
merged 3 commits into from
Nov 27, 2023

Conversation

JorgenG
Copy link
Contributor

@JorgenG JorgenG commented Oct 26, 2022

This is a:

  • bug fix PR with no breaking changes — please ensure the base branch is main
  • new functionality — please ensure the base branch is the latest dev/ branch
  • a breaking change — please ensure the base branch is the latest dev/ branch

Description & motivation

This PR switches from using the log function to using the print function. The main motivation for doing this is to enable piping the output from a codegen run operation command directly into the target file, instead of copy pasting.

Example:

dbt --quiet run-operation generate_model_yaml --args '{"model_name": "stg_jaffle_shop__orders"}' > models/staging/jaffle_shop/stg_jaffle_shop__orders.yml

This would also enable further automation, i.e. generating model yaml for all files in a directory etc.

Should resolve #56 and #71

Checklist

  • I have verified that these changes work locally
  • I have updated the README.md (if applicable)
  • Not applicable - I have added tests & descriptions to my models (and macros if applicable)
  • I have added an entry to CHANGELOG.md

@JorgenG JorgenG closed this Dec 14, 2022
@swiatek25
Copy link

@JorgenG why have you closed that PR? Is there any other way to achieve what print was supposed to deliver?

@JorgenG JorgenG reopened this Mar 17, 2023
@JorgenG
Copy link
Contributor Author

JorgenG commented Mar 17, 2023

@swiatek25 It seemed to be no interest in it and we have already skipped using the package and added our own implementation of this. The approach is valid, but niche. Outputting to file has it's challenges. Regardless, I'll reopen the PR if there seems to be interest in it.

@jenna-jordan
Copy link

Hello,

I have been struggling for days now to figure out how to output to a file instead of the standard output, and finally started investigating the codegen repo and found this PR. I would like to say that I am very interested in this feature, at the moment it is impossible to pipe output into a file (below is an example of what gets piped to the file instead):

�[0m17:40:24  Running with dbt=1.3.1
�[0m17:40:25  Encountered an error:
Parsing Error
  Error reading analytics: internal\stg\test.yml - Runtime Error
    unacceptable character #x001b: control characters are not allowed
      in "<unicode string>", position 0

So I just wanted to contribute my opinion that this would be a very desirable feature as the capability of outputting to a file would remove a lot of necessary manual effort when using codegen.

@swiatek25
Copy link

@jenna-jordan I worked around it by creating a wrapper macro that simply prints returned value from codegen macro.

{%- macro generate_model_metadata(model_name) -%}
    {% set model_yml = codegen.generate_model_yaml(model_names=[model_name]) %}
    {{ print(model_yml) }}
{%- endmacro %}

That is surely not perfect but considering I use a small subset of codegen macros it is acceptable.

Nevertheless I'm still voting for this PR.

@jenna-jordan
Copy link

@swiatek25 how would you call this macro as a dbt run operation within a shell script?

@swiatek25
Copy link

@swiatek25 how would you call this macro as a dbt run operation within a shell script?

dbt --quiet run-operation generate_model_metadata --args '{"model_name": "stg_jaffle_shop__orders"}' > output.file

note --quiet flag

@jaypeedevlin
Copy link
Contributor

@joellabes are you still looking after this repo, this seems like a good QOL addition (folks seem to be working around it right now).

Also https://github.com/dbt-labs/dbt-codegen/blob/main/dbt_project.yml#L4 needs to be bumped to >=1.1.0 to get the print feature in.

@joellabes joellabes merged commit e8b5a02 into dbt-labs:main Nov 27, 2023
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use print instead of log
5 participants