Log execution summary & restructure tests

• Emit execution id at the start of processing
• Emit execution summary at the end of processing
• House all test code within a ./tests directory

Schema changes

Adds

• #28
• #30
• #31

Changes

• Don't log data related errors unless using log level = DEBUG, instead log the failure reason to the data_load_execution table
• Add alembic so we can track schema changes
• Add an argument for the audit column prefix (breaking)
• Repackage and move tests

Add progress markers in logging

v0.1.4-beta

Merge pull request #29 from PageUpPeopleOrg/feature/OSC-1279-log-prog…

Add exta failover server compatibility

Includes:

• #27

Stop logging connection strings

v0.1.2-beta

Merge pull request #26 from PageUpPeopleOrg/feature/OSC-1259-do-not-l…

Add failover server compatibility

Includes PR #25

Example Usage

Before:

mssql+pyodbc://analytics_data_pipeline_543_user:hunter2@172.168.1.1/pp5_543?driver=SQL+Server+Native+Client+11.0

Now:

"mssql+pyodbc://analytics_data_pipeline_543_user:hunter2@(local)/pp5_543?driver=SQL+Server+Native+Client+11.0&failover=192.168.1.1"

note

• failover server is totally optional, and the old syntax works fine
• &failover requires the command to be quoted, as & is a special character
• no variables (ie username, password, host, driver, database, failover) can contain the following characters: @/?&:, this restriction could be made more permissive with some work

Added Audit Command

Change of commands:

• split up RDL into process/audit
• RDL process accepts a list of models to be force full refreshed

Extras:

• added unit testing
• linting
• repackaging
• minor bug fixes

Support for compound PKs

Merge pull request #6 from PageUpPeopleOrg/OSC-974_CompoundPKs

Added correct handling of compound primary keys

Add support for Boolean Data Type

Merge pull request #5 from PageUpPeopleOrg/AddSupportForBoolean

Added bool support

Incremental Loading

Merge pull request #4 from PageUpPeopleOrg/ExecutionLogs

Execution logs