Write "how to run Spoke tests locally" documentation

[lediur]

Scope of work

Research current process for getting tests to work in a local development environment, and document this process in README.md or CONTRIBUTING.md.

Motivation

Tests appear to require some non-obvious investigation and configuration to run on a freshly created local development environment.

Notes

Added some proposed fixes that seem like reasonable paths for more permanent fixes. Happy to discuss of course

• running yarn test directly is not successful. there are multiple failing test suites (13 / 29 suites failed, 11 / 111 tests failed)

  • suspected root cause: non-unit tests appear to be broken right now? I haven't investigated whether the tests are working, but CI doesn't appear to run integration or e2e tests.

  • workaround: use the test command specified in the CI pipeline:

    yarn jest --forceExit --detectOpenHandles src/**/*

  • short term fix: modify yarn test to use this command instead

  • long term fix: fix the other tests and make them pass 😜 apparently many of the e2e and integration tests are from 5 years ago

• tests fail to connect to database (connect ECONNREFUSED ::1:5432 where ::1 may be 127.0.0.1 or localhost depending on dev machine's network configuration)

  • root cause: the jest test runner uses a hardcoded DB_JSON configuration in jest.config.js that points to a user (spoke_test) and database (spoke_test) on a server running on port 5432. this configuration is not mentioned in setup documentation

  • workaround: modify jest.config.js to point to local development database server, and manually create separate spoke_test database to avoid contaminating application database

    # for example, with a server running on port 15432
    psql -U spoke -h localhost -p 15432 -d spokedev -c "create database spoke_test;"

  • short term fix: read the connection string from the environment (using a library like envalid or via process.env directly)

• tests timeout with SASL: SCRAM-SERVER-FIRST-MESSAGE: client password must be a string
  • root cause: multiple tests use the TEST_DATABASE_URL environment variable as their connection string, which is not provided in .env.example and not referred to in setup documentation. the environment variable is set to undefined by default, which causes the error
  • workaround: add TEST_DATABASE_URL to .env file pointing to the database
  • short term fix: add TEST_DATABASE_URL to .env.example and document in README. unset the default undefined configuration and throw an error if this environment variable is unset.
  • long term fix: use a common ROOT_DATABASE_URL and automatically generate ephemeral databases for testing
• tests fail to find different relations, even though codegen and migrations have run successfully

  • root cause: NODE_ENV is set to development, not test, so migrations run against the database defined in the DATABASE_URL environment variable, not the TEST_DATABASE_URL environment variable

  • workaround: set NODE_ENV to test, then run migrations. for example:

    NODE_ENV=test yarn migrate:worker && NODE_ENV=test yarn knex migrate:latest

  • short term fix: add a yarn migrate:test script that runs this line, and change documentation to fix

  • long term fix: run migrations as part of ephemeral test database setup