arbitrarily deep nesting

[drewbanin]

#50

Summmary

This branch primarily addresses code cleanup/refactoring. In addition, it seemed like a good time to add arbitrary model nesting support. Everything can be arbitrarily deeply nested. This includes .csv files, model .sql files, schema.yml files, and analysis files!

New dbt_project.yml model configurations

Suppose the root of your models repository looks like this:

dbt_project.yml
models/
├── other_service
│   ├── accounts.sql
│   ├── people.sql
│   ├── people_accounts.sql
│   └── schema.yml
└── saas_service
    └── summary
        └── domain_counts.sql
analysis/
└── domains
    └── domain_counts.sql

You can structure your models config in dbt_project.yml to selectively configure models at any depth you want. You must specify the full path to a model in the config for its configuration to take affect. That is, if your file is located at models/saas_service/summary/domain_counts.sql, your config should look like:

models:
  'Your Project Name':
    saas_service:
      summary:
        domain_counts:
          enabled: true

Of course, if you just want to enable all models in Your Project Name, you can write:

models:
  'Your Project Name':
    enabled: true

This works at any level of the config, and the most restrictive config that can apply to a model is used. Each level of the config "inherits" its parents config and can overwrite some or all of it.

A full example:

name: 'My Project Name'
version:  '1.0'
...
# default parameters that apply to _all_ models (unless overridden below)
model-defaults:
  enabled: true           # enable all models by default
  materialized: false     # If true, create tables. If false, create views

models:
  'My Project Name': # this is the name of your dbt project (listed above)
    enabled: true       # by default, enable models in this repository (including deps)
    saas_service:
      enabled: false     # disable 'My Project Name'.saas_service.*
      materialized: true
      emails:
        enabled: true # specifically enable 'My Project Name'.saas_service.emails

  'Analyst_Collective': # this is a dependency listed below
    enabled: false  # by default, disable all 'Analyst_Collective' models
    emails:
      enabled: true  # only turn on 'email' and 'pardot' models
    pardot:
      enabled: true

repositories:
   - "git@github.com:analyst-collective/analytics.git" # name = 'Analyst_Collective'