This connector extracts linage from a Snowflake account using Snowflake Connector. It queries ACCESS_HISTORY and therefore requires Snowflake Enterprise or higher.
Create a dedicated user & role based on the Setup guide for the general Snowflake connector.
The config file inherits all the required and optional fields from the general Snowflake connector Config File. In addition, you can specify the following configurations:
# (Optional) Whether to enable finding view lineage from object dependencies, default True.
enable_view_lineage: bool = True
# (Optional) Whether to enable finding table lineage information from access history and query history, default True.
enable_lineage_from_history: bool = True
# (Optional) Whether to include self-referencing loops in lineage, default False (exclude self loop)
include_self_lineage: <boolean>
# (Optional) Number of days to include in the usage analysis. Default to 7.
lookback_days: <days>
# (Optional) The number of access logs fetched in a batch, default to 100000
batch_size: <batch_size>Follow the Installation instructions to install metaphor-connectors in your environment (or virtualenv). Make sure to include either all or snowflake extra.
Run the following command to test the connector locally:
metaphor snowflake.linage <config_file>Manually verify the output after the run finishes.