-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix FuturesUnordered #105
Fix FuturesUnordered #105
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In terms of unit tests, I think the thing we're missing is polling the JoinHandle with a custom waker implementation -- things accidentally work today if the JoinHandle is polled with a waker provided by Shuttle, because it defers to the internal set_waiter
mechanism. So we want a unit test that somehow validates the waker is invoked when it's supposed to be.
block_on(async { | ||
let mut tasks = (0..100).map(|_| spawn(async move {})).collect::<FuturesUnordered<_>>(); | ||
|
||
while let Some(result) = tasks.next().await { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure this test really adds anything new, since the above collect essentially does the same thing, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, kinda, the internals of Next
and Collect
are a bit different. I think it is fine to have both.
Just a heads up that a fix with the double locking fixed and a test for the custom waker is coming. |
Hmm, it can be reapproved and merged now. The behaviour the test is intended to cover is currently exercised by FuturesUnordered through the existing tests. Can add the test later as we don't want to rely on FuturesUnordered internals. |
Need [this](awslabs/shuttle#105) fix, otherwise our concurrency tests are failing
Need [this](awslabs/shuttle#105) fix, otherwise our concurrency tests are failing
* Fix FuturesUnordered * Address comments from @jamesbornholt and @jorajeev * Remove double locking in future/mod.rs
Currently, code using FuturesUnordered has a chance to cause Shuttle to erroneously report a deadlock.
For instance, if we take the following code:
When we call
next()
on a task, that task gets dequeued from theready_to_run_queue
insideFuturesUnordered
. If that task isPoll::Ready
, then all is fine. If it isPoll::Pending
(in the case where we callnext()
before thespawn
has had a chance to execute) however, then the task is never polled again, as we don't wake the associated waker.This PR seeks to solve this issue by waking the waker, which is stored together with the result in the
Wrapper
.Currently the test suite is a bit lacking, and only contains three tests of common patterns with futures that don't do anything. I have some more tests locally, but they are really just reskins of existing tests, so I am not entirely sure they check any interesting behaviour. I want to add some more interesting tests, so going to look at that next. If anyone has any suggestions, then I am all ears. Also putting this out here to get feedback on the solution in general.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.