Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: disable integration tests #2062

Merged
merged 1 commit into from
May 30, 2023
Merged

CI: disable integration tests #2062

merged 1 commit into from
May 30, 2023

Conversation

julianoes
Copy link
Collaborator

Some of the tests are more often failing than passing and we stopped even checking the results. So we might as well disable the tests for now, so that it at least looks better in the README badge and PR checks.

Also, we save some of Azures CPU cycles just burns energy without any gain at this point.

I think in the future we can discuss bringing back some tests, or just have some examples executable as tests instead but heavily rely on the system-tests instead.

The example tests added back in would be to test a few cases that cover whether things are roughly still connecting and running, however, it won't be feasible to have integration tests for all sorts of combinations of PX4 and ArduPilot versions against all of the various features because we just run into combinatorial explosion.

Some of the tests are more often failing than passing and we stopped
even checking the results. So we might as well disable the tests for
now, so that it at least looks better in the README badge and PR checks.

Also, we save some of Azures CPU cycles just burns energy without any
gain at this point.

I think in the future we can discuss bringing back some tests, or just
have some examples executable as tests instead but heavily rely on the
system-tests instead.

The example tests added back in would be to test a few cases that cover
whether things are roughly still connecting and running, however, it
won't be feasible to have integration tests for all sorts of
combinations of PX4 and ArduPilot versions against all of the various
features because we just run into combinatorial explosion.

Signed-off-by: Julian Oes <julian@oes.ch>
Copy link
Collaborator

@JonasVautherin JonasVautherin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense 👍

@JonasVautherin JonasVautherin merged commit 5eaf2cc into main May 30, 2023
@JonasVautherin JonasVautherin deleted the pr-disable-sitl-tests branch May 30, 2023 09:23
@julianoes julianoes mentioned this pull request Dec 27, 2023
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants