Handle "meta:" key in schema/sources.yml

[tayloramurphy]

Closes #1362

This PR adds the ability for people to use the key meta: to add arbitrary information to their schema.yml and sources.yml files.

I have tested locally by having this in my schema.yml

version: 2
models:
  - name: sfdc_account
    description: base model for SFDC Accounts tayloramurphy
    meta:
      owned: "@tayloramurphy"
    columns:
      - name: account_id
        meta:
          sensitivity: 1

and in my sources.yaml

version: 2

sources:
  - name: salesforce
    database: '{{ env_var("SNOWFLAKE_LOAD_DATABASE") }}'
    schema: salesforce_stitch
    loader: Stitch
    loaded_at_field: _sdc_batched_at
    meta:
      owner: "@tayloramurphy_source_data"
    
    quoting:
      database: true
      schema: false
      identifier: false

    freshness:
        warn_after: {count: 8, period: hour}
        error_after: {count: 24, period: hour}

    tables: 
      - name: account
        description: '{{ doc("sfdc_account_source") }}'
        meta:
          sensitivity: "@tayloramurphy_source_data_table"

And was able to see in the manifest.json the data written appropriately. dbt also ran just fine.

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Taylor A. Murphy.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[tayloramurphy]

I just signed the CLA.

@beckjake @drewbanin do I need to add anything to the tests for this? I saw several schema.yml files in the integration tests section but wasn't sure where this would best fit in.

[drewbanin]

Thanks for opening this PR @tayloramurphy! (@cla-bot check)

I just added a comment to the discussion on naming this field in the issue. Let me know what you think about that suggestion in the issue thread.

I'll check out the code changes here. Kicking off the tests presently.

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Taylor A. Murphy.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!

[tayloramurphy]

I see the failing tests - will work on updating.

[tayloramurphy]

@drewbanin pg integration tests are failing but it looks like it's on the install postgres step. I tried submitting another commit to see if it was a one-off but maybe there's something deeper? Please advise.

[drewbanin]

@tayloramurphy this is definitely weird - I see:

[NuGet] The operation has timed out

It looks like it's related to an incident over here: https://status.chocolatey.org/

I'll kick off these tests again when it looks like things have been resolved w/ chocolatey

[beckjake]

I saw this happen for a few days in a row with another PR, but could not repro on my Windows box. It mysteriously went away after a time - I guess chocolatey is having some problems.

[drewbanin]

/azp run

[tayloramurphy]

I'll get back to this after the Christmas holiday!

[tayloramurphy]

@drewbanin looks like postgres integration tests are passing but AFAICT the BQ, Snowflake, and Redshift integration tests are failing b/c it can't connect. Is this still related to the chocolately outage? https://status.chocolatey.org/issues/2019-12-18-timeouts-when-searching-and-installing-packages/

[tayloramurphy]

Yes! Tests are passing! @drewbanin or @beckjake officially submitting for review 😄

[drewbanin]

Thanks @tayloramurphy! I gave this a quick functional test and it looks really good to me :)

I also just queued up this issue to render these eventual values in the docs site: dbt-labs/dbt-docs#66

Nice work with the tests here - I know that updating node contracts in the test suite can be a real slog!

Adding @beckjake for code review

[beckjake]

looks great!

[drewbanin]

Just merged - thanks for your contribution to dbt @tayloramurphy :D

[tayloramurphy]

Thank you both for the review! I'm pumped to get this feature in 😄