deprecate sql_where

[drewbanin]

See also:

The sql_where config isn't actually useful and people shouldn't use it. We should:

• add a deprecation warning when sql_where is provided
• make sql_where no longer required
• revamp docs around incremental models (Update incremental model docs to explain how and when to use sql_where (never) #1036)
• make it easier/faster to do the "advanced" version of incremental models

[drewbanin]

@jthandy I'm thinking about a better way to handle incremental models. I think that the general pattern in place currently is a good one, but we could make it a lot simpler with a macro.

I figure we can provide this is_incremental() macro in dbt. The macro can then be used in projects as shown below:

{% macro is_incremental() %}
    {#-- do not run introspective queries in parsing #}
    {% if not execute %}
        {{ return(False) }}
    {% else %}
        {% set relation = adapter.get_relation(this.schema, this.table) %}
        {{ return(relation is not none and not flags.FULL_REFRESH) }}
    {% endif %}
{% endmacro %}

-- models/some_model.sql

{{ config(materialized='incremental') }}

select * from public.events

{% if is_incremental() %}
    where dvce_created_tstamp > (select max(dvce_created_tstamp) from {{ this }})
{% endif %}

This is just some sugar over the existing implementation, but it also helps dbt avoid doing any queries at parse-time

Very happy to iterate on this -- keen to hear your thoughts!

cc @clrcrl

[jthandy]

i just wanted to get people to stop using a bad feature we built in 2016 so this is beyond my wildest dreams :D I have literally nothing to add to this--love it.

[clrcrl]

So as I mentioned to @drewbanin in person, I also love this!
The only thing I don't love is the terminology is_incremental – I would expect that to be true for any model which has materialized == 'incremental'.
I don't have a better term to offer at present... is_incremental_run? Maybe? Eh.

[drewbanin]

I'm with you @clrcrl -- is_incremental() is kind of funky. I played around with partial_run() or insert_new() or apply_filter() but all of those are contextless IMO. I think we should just pick one -- if users have strong opinions, they can alias the name with their own macro :)