Refactor run_tests.sh script #6280
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #6280 +/- ##
==========================================
- Coverage 26.09% 9.82% -16.28%
==========================================
Files 32 32
Lines 2698 2698
==========================================
- Hits 704 265 -439
- Misses 1948 2411 +463
+ Partials 46 22 -24
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
tests/e2e/scripts/run_tests.sh
Outdated
| } | ||
|
|
||
| ls createreport/rke2_"$OS".log 2>/dev/null && rm createreport/rke2_"$OS".log | ||
| ls createreport/rke2_${date}.log 2>/dev/null && rm createreport/rke2_${date}.log |
There was a problem hiding this comment.
Can we leave it as OS to avoid cleanup of s3 bucket and preferably one sharable link to results for different OS?
There was a problem hiding this comment.
ok, done! I moved all tests to Ubuntu2310 (same like in k3s)
|
|
||
| echo 'RUNNING MIXEDOS TEST' | ||
| /usr/local/go/bin/go test -v mixedos/mixedos_test.go -nodeOS="$nodeOS" -serverCount=$((servercount)) -timeout=1h -json -ci |tee -a createreport/rke2_"$OS".log | ||
| for i in ${!tests[@]}; do |
There was a problem hiding this comment.
Some tests have just server nodes and no agents. Some have HA and some single server single agent. Can we default then in Vagrantfile?
There was a problem hiding this comment.
Yes, vagrantfile defaults the number of servers and agents per test. Same for the OS
Signed-off-by: Manuel Buil <mbuil@suse.com>
manuelbuil
force-pushed
the
improveruntestssh
branch
from
0f3ab8f
to
46e2577
Compare
Signed-off-by: Manuel Buil <mbuil@suse.com>
manuelbuil
force-pushed
the
improveruntestssh
branch
from
46e2577
to
fdc40dd
Compare
* Improve run_tests script Signed-off-by: Manuel Buil <mbuil@suse.com> * Upgrade tests to ubuntu2310 and use nodeOS var Signed-off-by: Manuel Buil <mbuil@suse.com> --------- Signed-off-by: Manuel Buil <mbuil@suse.com>
Proposed Changes
This PR adds some extra logging in the latest_commit.sh script. If there was an error because of the API exceeding the limits, it was impossible to notice
Moreover, it refactors run_tests.sh script trying to make it simpler. Changes:
1 - Removes all the parameters that can be passed except for nodeOS, which is important for the results generation. Some of them depend on the test itself and should not be changed (servercount, agentcount). Some are not read by any test (db or hardened)
2 - Removes
rke2_versionandrke2_channelvariables and references as the former was always empty and the latter was alwayscommit, which is anyway the default: https://github.com/rancher/rke2/blob/master/tests/e2e/vagrantdefaults.rb#L19-L223 - Creates a loop that goes over an array with the different e2e tests available instead of calling each one in a different line
4 - Uses a 2h timeout for all e2e tests. In the end, the important timeout is the one happening in each Test Case (in the Eventually instruction)
5 - Slightly changes the "VM gargabe collection". It was detected that sometimes, VMs are not really cleaned up. That should always happen as we run the tests with the
-ci. We suspect it's a problem related to the timeout, that's why we increased it to 2h as previously explained. Just in case, before starting the e2e test, we destroy the VMs related to that test. At the beginning of the script, we also destroy any VM which might be in an invalid state withvagrant global-status --prune6 - The log file includes now the date in the name instead of the OS
7 - Upgrades all images to Ubuntu2310
Before running each test, it makes sure that VMs are destroyed.
Types of Changes
Code refactor
Verification
Testing
Linked Issues
User-Facing Change
Further Comments