MaveDB Local Development

Project Setup

To develop on MaveDB, it's easiest to utilize the Docker containers maintained with the project. To get set-up for development, first download Docker Desktop.

MaveDB also relies on an external dependency to dcd-map, a project that maps MaveDB variants to interoperable variant objects which are relevant to the human genome. You should clone the mapping repository, checkout the mavedb-main branch, and build the container. The dev version of MaveDB expects this docker container to have the tag dcd-mapping:dev.

git clone https://github.com/VariantEffect/dcd_mapping2.git
cd dcd_mapping2
git checkout mavedb-main
docker build --tag dcd-mapping:dev .

With this external dependency taken care of, you are now ready to start the development environment for MaveDB. Simply clone the repository and compose the Docker project:

git clone https://github.com/VariantEffect/mavedb-api.git
cd mavedb-api
docker-compose -f docker-compose-dev.yml up --build -d

The project will reload code changes within the application's development environment automatically, so there is no need to reload the application when making changes to it. Note that the same does not apply to the worker or mapping environments-- these will need to be rebuild if any changes are made to them.

To create the MaveDB environment, simply run poetry install --extras server --with dev within the project's root directory. This will install the project's dev and server dependencies and provide a way for you to run scripts or tools related to the project. You should note that when running commands or scripts in the development environment, our default log configuration attempts to spawn a watchtower process that will cause these commands to fail unless run with a specific environment variable. Watchtower is a logging package that allows us to easily send application logs to AWS CloudWatch. To ensure local commands do not attempt to spawn this process, make sure the environment variable LOG_CONFIG is set to one of test or dev.

Database Migrations

MaveDB uses alembic to handle database migrations. Changing the models directly will allow alembic to detect database changes and auto-generate revisions that, when run, will change the state of the database to match the defined models. To auto-generate these revisions, simply run the command below while the development database is running

LOG_CONFIG=dev DB_PORT=5434 DB_DATABASE_NAME=mavedb DB_USERNAME=postgres DB_PASSWORD=postgres poetry run alembic revision --autogenerate -m "<message>"

To apply the most up-to-date version of the schema to the development database, run

LOG_CONFIG=dev DB_PORT=5434 DB_DATABASE_NAME=mavedb DB_USERNAME=postgres DB_PASSWORD=postgres poetry run alembic upgrade head

And to downgrade the development database schema by a particular number of revisions, simply run

LOG_CONFIG=dev DB_PORT=5434 DB_DATABASE_NAME=mavedb DB_USERNAME=postgres DB_PASSWORD=postgres poetry run alembic downgrade -<# of revisions>

Linting and Formatting

This project uses Ruff for linting and formatting. If you use VSCode, consider the Ruff extension from the VS Code Marketplace.

To lint code prior to committing, run

poetry run ruff check

Certain issues might be fixed automatically via

poetry run ruff check --fix

To format code, run

poetry run ruff format

Note that this command is equivalent to running formatting on the entire current directory and any subdirectories. For more granular formatting, pass the format command the path explicitly.

For additional features and details, you can access the Ruff documentation

Testing

MaveDB uses MyPy to validate type hints within the project and Pytest to run our unit test suite. You can run these checks locally with the following commands:

MyPy:

LOG_CONFIG=dev poetry run mypy src/

Pytest:

LOG_CONFIG=dev poetry run pytest tests/ --cov=src