-
Notifications
You must be signed in to change notification settings - Fork 411
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Retry flaky tests #822
Retry flaky tests #822
Conversation
CI failure due to: ipython/ipython#13554 |
noxfile.py
Outdated
@@ -142,6 +150,8 @@ def test_all_packages(session): | |||
"--show-capture=no", | |||
"--net-pypiserver", | |||
"--all-packages", | |||
"--reruns=5", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this many? I’d start with 2.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done!
Before we go to retries, we should look to understand the root cause of the problem. I would rather hold off on landing this before we try other methods, since this is adding more tech debt. I appreciate the PR and if you can’t root cause it, that’s fine, but I’d prefer creating an issue instead and closing this PR, especially because CI tests still failed. |
Basically, iPython needs to release the fix ipython/ipython#13554 (comment) I think retries are still a useful thing to add, though it's up to you! |
From experience, subprocess (also |
We have had zero of such issues with pipx so far. I just looked into the failures we're seeing a little, and the root cause is a repeatable python3.8/ipython issue. ipython/ipython#12677 I appreciate the PR, but I am going to close it since I believe it is trying to solve a problem that does not exist, and is adding tech debt. |
Motivation: #817 (comment)