Fix publishing of coverage results to coveralls.io (+ misc test config updates) #890
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Coveralls is used to publish coverage results online via coveralls.io. Travis is already configured to run it "after_success", but this has failed for a while, because it was not installed. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
- Add a build matrix to run each tox env in a corresponding travis env as per travis/tox best practices. https://docs.travis-ci.com/user/languages/python/#using-tox-as-the-build-script - Add Python 3.5 tests - Remove only build on certain branch restrictions - Use "install" instead of "before_script" to install dependencies. Explicitly listing "install" prevents Travis from automatically running `pip install -r requirements.txt`, which is not necessary because most of those requirements are installed again in each tox environment. - Move pylint and bandit calls to tox (pylint requires dependencies) to be installed. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
|
|
We install coverage inside tox builds to generate test coverage reports. These reports need to be created with a version supported by coveralls, which we use (outside of tox) to publish coverage reports to coveralls.io. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
lukpueh
force-pushed
the
fix-coveralls
branch
3 times, most recently
from
62015c6
to
b2dd8e9
Compare
This replicates behavior of unittest's `discover` method, and allows `coverage` and the tool that posts coverage reports to coveralls.io, i.e. `coveralls`, to record the correct paths and left-strip the parts leading to the project directory. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
lukpueh
force-pushed
the
fix-coveralls
branch
2 times, most recently
from
4fdc395
to
157167e
Compare
|
Fixed! See most recent Travis builds and corresponding coverage reports on coveralls. The badge on the main README should be updated once this PR is merged and IIUC we will see GitHub status (as in the table at the bottom) in subsequent PRs, because coveralls needs a reference build. |
3 tasks
tanishqjasoria
approved these changes
Jul 8, 2019
This commit partially reverts the workaround introduced by 157167e. Instead of patching the path, we configure tox to install TUF in editable mode, which makes the tests run against the same files as if the path were patched. This makes it so that coverage records paths that it can then normalize when sending them to coveralls.io (see .travis.yml). See uptane/obsolete-reference-implementation@af22701 for detailed background information. As a consequence we can now skip building of sdist and installing it into a virtual env directory in tox. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Coverage used to be configured to omit certain directories while reporting. This commit slightly optimizes coverage to already omit those directories while measuring coverage. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
adityasaky
suggested changes
Sep 10, 2019
.travis.yml
Outdated
| # FIXME: Consider refactoring the tests to not require the test aggregation | ||
| # script being invoked from the `tests` directory, so that `.coverage` is | ||
| # written to and .coveragrc can also reside in the project root directory, as | ||
| # as the convention. |
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
|
Thanks, @adityasaky! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes issue #:
None
Description of the changes being introduced by the pull request:
Code coverage results from our Travis builds have not been published to coveralls.io in a while, due to the required
coverallspublishing tool not being properly installed.Besides fixing publishing of coverage results, this PR cleans up/ updates other testing configurations:
https://docs.travis-ci.com/user/languages/python/#using-tox-as-the-build-script
Explicitly listing "install" prevents Travis from automatically running
pip install -r requirements.txt, which is not necessary because most of those requirements are installed again in each tox environment.Please verify and check that the pull request fulfills the following
requirements: