Skip to content

Commit

Permalink
Updating freshness ref page (#4770)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Updating the `freshness` ref page to include new information for v1.7

Closes #4615

## Checklist

- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [x] For [docs
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning),
review how to [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
- [x] Add a checklist item for anything that needs to happen before this
PR is merged, such as "needs technical review" or "change base branch."
  • Loading branch information
matthewshaver authored Jan 19, 2024
2 parents dc9ac9a + 6a8ee06 commit 4d46b20
Showing 1 changed file with 33 additions and 1 deletion.
34 changes: 33 additions & 1 deletion website/docs/reference/resource-properties/freshness.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,38 @@ A freshness block is used to define the acceptable amount of time between the mo

In the `freshness` block, one or both of `warn_after` and `error_after` can be provided. If neither is provided, then dbt will not calculate freshness snapshots for the tables in this source.

<VersionBlock firstVersion="1.7">

In most cases, the `loaded_at_field` is required. Some adapters support calculating source freshness from the warehouse metadata tables and can exclude the `loaded_at_field`.

If a source has a `freshness:` block, dbt will attempt to calculate freshness for that source:
- If a `loaded_at_field` is provided, dbt will calculate freshness via a select query (behavior prior to v1.7).
- If a `loaded_at_field` is _not_ provided, dbt will calculate freshness via warehouse metadata tables when possible (new in v1.7 on supported adapters).

Currently, calculating freshness from warehouse metadata tables is supported on:
- [Snowflake](/reference/resource-configs/snowflake-configs)

Support is coming soon to the following adapters:
- [Redshift](/reference/resource-configs/redshift-configs)
- [BigQuery](/reference/resource-configs/bigquery-configs)
- [Spark](/reference/resource-configs/spark-configs)

Freshness blocks are applied hierarchically:
- a `freshness` and `loaded_at_field` property added to a source will be applied to all all tables defined in that source
- a `freshness` and `loaded_at_field` property added to a source _table_ will override any properties applied to the source.

This is useful when all of the tables in a source have the same `loaded_at_field`, as is often the case.

To exclude a source from freshness calculations, you have two options:
- Don't add a `freshness:` block.
- Explicitly set `freshness: null`.

## loaded_at_field
(Optional on adapters that support pulling freshness from warehouse metadata tables, required otherwise.)
</VersionBlock>

<VersionBlock lastVersion="1.6">

Additionally, the `loaded_at_field` is required to calculate freshness for a table. If a `loaded_at_field` is not provided, then dbt will not calculate freshness for the table.

Freshness blocks are applied hierarchically:
Expand All @@ -47,7 +79,7 @@ This is useful when all of the tables in a source have the same `loaded_at_field

## loaded_at_field
(Required)

</VersionBlock>
A column name (or expression) that returns a timestamp indicating freshness.

If using a date field, you may have to cast it to a timestamp:
Expand Down

0 comments on commit 4d46b20

Please sign in to comment.