Sonarcloud with code coverage

[jamshale]

This is used to get code coverage for our sonarcloud integration on PR's and when code is pushed to main.

The complication comes with using the sonarcloud token from forked repos. I have tested using two github accounts and forking my fork.

This works by saving the test report from the PR Tests workflow as an artifact and also the pr number. Then when the PR Tests workflow completes successfully a workflow will begin for sonarcloud witch gets information about the forked repo, and fetches the code for the scan. The code coverage is downloaded from the artifact, updated and then upload to sonarcloud.

Another workflow runs the tests on merge to main, runs the scan and uploads results. I thought about doing this as one workflow but decided separate workflows was actually much simpler.

I changed the workflow Tests to an action and adjusted the other workflows that used it. When calling from the push to main workflow it was having trouble as a workflow.

Requires disabling Automatic Analysis in the sonarcloud admin console. You can't use automatic and manage it from CI at the same time. Also won't exist until the workflows have been merged to main.

Main branch:

PR (Failing):

The required coverage can be adjusted in sonarcloud console.

PR (Passing):

[jamshale]

Hmm. Might have messed something up with that Tests / Test Python 3.9 Expected — Waiting for status to be reported

I think it could just be changed to PR Tests, or I could change PR Tests to take an input. I think Tests workflow should still be an action instead of a workflow.

[WadeBarnes]

A few questions, and a suggestion.

Is it necessary to run the sonar scan on the PR and on the merge? Just wondering what the difference is and whether running the scan on merge would be redundant since it's already been run on the PR.

[WadeBarnes]

We'll need to update the Required checks to switch to the updated test flow/action. If we do that before we merge this PR, it will have to be done right before the merge, as the change will impact the required checks on all outstanding PRs, and they will need to be updated so they run and pass the new test flow/action.

[jamshale]

A few questions, and a suggestion.

Is it necessary to run the sonar scan on the PR and on the merge? Just wondering what the difference is and whether running the scan on merge would be redundant since it's already been run on the PR.

I think it's necessary to get the code coverage on main and not just the PR. Also, I think sonarcloud is already doing a scan on both currently, when using automatic analysis. I can have a look and see if I can find any information on this.

[jamshale]

I may be able to use the same code coverage artifact to avoid needing to run the tests on merge to main. I will look at doing that.

For the extra scan on merge to main I don't think anything needs to be changed. This is the same behaviour as automatic analysis after reading the sonarcloud documentation.

[WadeBarnes]

I may be able to use the same code coverage artifact to avoid needing to run the tests on merge to main. I will look at doing that.

For the extra scan on merge to main I don't think anything needs to be changed. This is the same behaviour as automatic analysis after reading the sonarcloud documentation.

Sounds good. Let me know when your ready for final review.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[jamshale]

@WadeBarnes So, I tried to use the previous test coverage artifact but have been having a hard time locating it from this workflow. It's a lot different than the other workflow which has the workflow run id. I thought I could use the pr commit, but haven't been able to make it work.

It would be nice to not run the tests twice for the same code, but I also don't want to mess around with this for a long time. So, I think this should be ready and maybe I'll be able to figure it out for a later PR.

[WadeBarnes]

Branch protection checks have been updated to account for the changes made to the workflows in this PR. This will impact ALL existing PRs, which will have to be updated to run the updated workflows in order to pass the branch protection checks.

cc @jamshale, @swcurran, @dbluhm

[WadeBarnes]

Failure here was my fault. I forgot to turn off automatic analysis. It's off now.

[WadeBarnes]

Job was rerun successfully.