Tokio Porting #566
Merged
Tokio Porting #566
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
YuanYuYuan
force-pushed
the
PR/tokio-porting
branch
from
dc4d0df
to
4821577
Compare
Closed
|
Hi @YuanYuYuan |
|
Any update if this PR is till active? |
Oops, sorry I missed this message. Let me mark it as WIP in the title and convert it to a Draft PR. |
Sure. We've been testing the performance on this porting. FYI, this is on our roadmap for the 1.0.0 release. |
|
Any chance to issue a pre-release RC so that I can test it?
Also, is there an ETA for the 1.0, release?
…On Thu, Dec 14, 2023 at 4:04 PM YuanYuYuan ***@***.***> wrote:
Any update if this PR is till active?
Sure. We've been testing the performance on this porting. FYI, this is on
our roadmap for the 1.0.0 release.
—
Reply to this email directly, view it on GitHub
<#566 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFYR7XFRNTLTYNADUHXXVMDYJKXIXAVCNFSM6AAAAAA53XYGL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJVGM3DEMZRHE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
This PR is not ready yet to be merged and a pre-release is not planned. |
|
I've moved away from Zenoh to Fluvio.io to progress my project therefore I can't allocate time to tinker around with an untested PR. |
|
@YuanYuYuan Please change your pull request's base branch to main (new default branch). And rebase your branch against |
This was referenced Jan 22, 2024
Closed
This was referenced Jan 31, 2024
Closed
YuanYuYuan
force-pushed
the
PR/tokio-porting
branch
from
1bd09c3
to
8c0b223
Compare
YuanYuYuan
force-pushed
the
PR/tokio-porting
branch
2 times, most recently
from
2f226d3
to
3733c8b
Compare
YuanYuYuan
force-pushed
the
PR/tokio-porting
branch
from
39befc3
to
15f11b9
Compare
Merged
YuanYuYuan
added a commit
to YuanYuYuan/zenoh
that referenced
this pull request
Mar 14, 2024
* Tokio porting in zenoh-link * Porting further for transport * Work around mixed runtime problem in async drop * No need to await for tcp addresses anymore * Use tokio runtime for the TX worker * Replace all underlying runtime to tokio * Tidy codes * Add WIP lowlatency * Finished the porting of pipelien. Note the usage of spawn_blocking * Tokio porting on unixpipe * Use async_global_executor::block_on * work around the potential blocking task in unixpipe * Replace Async<File> * Replace Async<File> by AsyncFd<File> * Fix the runtime problem while dropping * Use ZRuntime. Note that low_latency_transport doesn't work * Test with dashmap. There seems to be a deadlock while P2P scouting * 1. Remove unneccessary async to solve block_on 2. Set non-blocking in UDP socket to use tokio UdpSocket * Address block_on issues in unixpipe and low-latency transport * Fix bug * Use deref to call ZRuntime and tidy up unused imports * Try with interval * Use TX runtime only * Ignore files starting with underscore * Use all ZRuntimes * Fix the busy loop in unixpipe * Unifty the ztimeout macro and remove the zasync_executor_init * Port zenoh/tests to tokio * Fix the nested block_on issue * Remove trailing whitespaces * Fix the tests in io/zenoh-transport * Remove async-std in zenoh/zenoh * Remove async-std in zenohd * WIP on tests of I/O * Pass all tests in io and zenoh * cargo fmt * remove unused dashmap * Finish the porting on examples * Make from_config async again * Use tokio::net::lookup_host to convert the address to socket addresses asynchronously * Porting on zenoh-link-ws * Cargo fmt * Remove commented lines * Tokio porting on serial * Tidy up * Add a throughput quick test script * Tokio porting on udp * Update Cargo.lock * Correct the comment * Correctly return the result * Porting on zenoh-link-unixsocket_stream * Correct the feature flag of tokio-util * Port link-tls to tokio and bump workspace rustls to 0.22.2 * Port link-quic to tokio * Turn get_listeners and get_locators into async calls. Use AsyncMutex and AsyncRwLock explicitly * Tidy up ZRuntimes * Tokio porting on zenoh-plugin-storage-manager * Refine ZRuntime * Enable the std feature of zenoh-result to convert std::io::Result * Tokio porting on zenoh-sync * tokio porting on object_pool * tokio porting on plugins * tokio porting on zenoh-ext * Upgrade a few dependencies * Tokio porting on doc * Add the missing io trait * Skip those troublesome tests * Remove the unneeded ASYNC_STD_THREAD_COUNT * Cargo fmt * Use the environmental variable to configure ZRuntime * Cargo fmt * Tidy up tokio features * Add the std feature used in zenoh-result * Fix matching_status_* failures * Speed up routing test and work around the failure of three_node_combination test * Halve the chunk size * Cargo clippy * Prevent the race condition * Turn three_node_combination into a sequential test * Introduce CancellationToken to terminate anyway * It turns out to be the large size payload fail the test. * Fix deadlock by reintroducing five ZRuntimes * Revert the tokio-porting on plugins since it cause a deadlock in routing test * Adjust the ZRuntime properly to fix the deadlock in routing test * Align plugins to the main as we would skip them in tokio porting * Apply Luca's patch of object_pool to make it sync only * Fix bug in matching * Cargo clippy * Add back TimedEvent to be compatible with plugins * Fix tests * Quickfix for plugins * Rename token name * Remove the unused Signal * Introduce TaskTracker and CancellationToken into zenoh-link-tcp * Let's use ZRuntime::Reception to handle accept_task * Refine the testing scripts * Let CI decide the minimal number of ZRuntime threads * Make it easier to pass the test * Revert "Let CI decide the minimal number of ZRuntime threads" since this fail This reverts commit de5c757. * Cargo fmt & clippy * Make cargo nextest more resilient * Cargo fmt & clippy & doc test * Add the missing Windows snippet * Fix after rebase * Remove the unneeded async-std * Rename ZRuntime::Reception to ZRuntime::Acceptor * Refine the PR * Use tokio::time and slow down the message rate to 500 Hz in the downsampler test * Fix tokio::time and use tokio socket bind_device * Fix cargo clippy * Cargo fmt * Add the warning of 2ms resolution issue of tokio * Address the unused variables * Correct the error types * Address the platform compiling issue * Use warning while binding the interface * Fix cargo clippy * Use JoinHandle to manage the listening task * Refine the TODO comments --------- Co-authored-by: Luca Cominardi <luca.cominardi@gmail.com>
YuanYuYuan
added a commit
to YuanYuYuan/zenoh
that referenced
this pull request
Mar 14, 2024
* Tokio porting in zenoh-link * Porting further for transport * Work around mixed runtime problem in async drop * No need to await for tcp addresses anymore * Use tokio runtime for the TX worker * Replace all underlying runtime to tokio * Tidy codes * Add WIP lowlatency * Finished the porting of pipelien. Note the usage of spawn_blocking * Tokio porting on unixpipe * Use async_global_executor::block_on * work around the potential blocking task in unixpipe * Replace Async<File> * Replace Async<File> by AsyncFd<File> * Fix the runtime problem while dropping * Use ZRuntime. Note that low_latency_transport doesn't work * Test with dashmap. There seems to be a deadlock while P2P scouting * 1. Remove unneccessary async to solve block_on 2. Set non-blocking in UDP socket to use tokio UdpSocket * Address block_on issues in unixpipe and low-latency transport * Fix bug * Use deref to call ZRuntime and tidy up unused imports * Try with interval * Use TX runtime only * Ignore files starting with underscore * Use all ZRuntimes * Fix the busy loop in unixpipe * Unifty the ztimeout macro and remove the zasync_executor_init * Port zenoh/tests to tokio * Fix the nested block_on issue * Remove trailing whitespaces * Fix the tests in io/zenoh-transport * Remove async-std in zenoh/zenoh * Remove async-std in zenohd * WIP on tests of I/O * Pass all tests in io and zenoh * cargo fmt * remove unused dashmap * Finish the porting on examples * Make from_config async again * Use tokio::net::lookup_host to convert the address to socket addresses asynchronously * Porting on zenoh-link-ws * Cargo fmt * Remove commented lines * Tokio porting on serial * Tidy up * Add a throughput quick test script * Tokio porting on udp * Update Cargo.lock * Correct the comment * Correctly return the result * Porting on zenoh-link-unixsocket_stream * Correct the feature flag of tokio-util * Port link-tls to tokio and bump workspace rustls to 0.22.2 * Port link-quic to tokio * Turn get_listeners and get_locators into async calls. Use AsyncMutex and AsyncRwLock explicitly * Tidy up ZRuntimes * Tokio porting on zenoh-plugin-storage-manager * Refine ZRuntime * Enable the std feature of zenoh-result to convert std::io::Result * Tokio porting on zenoh-sync * tokio porting on object_pool * tokio porting on plugins * tokio porting on zenoh-ext * Upgrade a few dependencies * Tokio porting on doc * Add the missing io trait * Skip those troublesome tests * Remove the unneeded ASYNC_STD_THREAD_COUNT * Cargo fmt * Use the environmental variable to configure ZRuntime * Cargo fmt * Tidy up tokio features * Add the std feature used in zenoh-result * Fix matching_status_* failures * Speed up routing test and work around the failure of three_node_combination test * Halve the chunk size * Cargo clippy * Prevent the race condition * Turn three_node_combination into a sequential test * Introduce CancellationToken to terminate anyway * It turns out to be the large size payload fail the test. * Fix deadlock by reintroducing five ZRuntimes * Revert the tokio-porting on plugins since it cause a deadlock in routing test * Adjust the ZRuntime properly to fix the deadlock in routing test * Align plugins to the main as we would skip them in tokio porting * Apply Luca's patch of object_pool to make it sync only * Fix bug in matching * Cargo clippy * Add back TimedEvent to be compatible with plugins * Fix tests * Quickfix for plugins * Rename token name * Remove the unused Signal * Introduce TaskTracker and CancellationToken into zenoh-link-tcp * Let's use ZRuntime::Reception to handle accept_task * Refine the testing scripts * Let CI decide the minimal number of ZRuntime threads * Make it easier to pass the test * Revert "Let CI decide the minimal number of ZRuntime threads" since this fail This reverts commit de5c757. * Cargo fmt & clippy * Make cargo nextest more resilient * Cargo fmt & clippy & doc test * Add the missing Windows snippet * Fix after rebase * Remove the unneeded async-std * Rename ZRuntime::Reception to ZRuntime::Acceptor * Refine the PR * Use tokio::time and slow down the message rate to 500 Hz in the downsampler test * Fix tokio::time and use tokio socket bind_device * Fix cargo clippy * Cargo fmt * Add the warning of 2ms resolution issue of tokio * Address the unused variables * Correct the error types * Address the platform compiling issue * Use warning while binding the interface * Fix cargo clippy * Use JoinHandle to manage the listening task * Refine the TODO comments --------- Co-authored-by: Luca Cominardi <luca.cominardi@gmail.com>
Closed
Mallets
added a commit
that referenced
this pull request
Mar 14, 2024
* Tokio Porting (#566) * Tokio porting in zenoh-link * Porting further for transport * Work around mixed runtime problem in async drop * No need to await for tcp addresses anymore * Use tokio runtime for the TX worker * Replace all underlying runtime to tokio * Tidy codes * Add WIP lowlatency * Finished the porting of pipelien. Note the usage of spawn_blocking * Tokio porting on unixpipe * Use async_global_executor::block_on * work around the potential blocking task in unixpipe * Replace Async<File> * Replace Async<File> by AsyncFd<File> * Fix the runtime problem while dropping * Use ZRuntime. Note that low_latency_transport doesn't work * Test with dashmap. There seems to be a deadlock while P2P scouting * 1. Remove unneccessary async to solve block_on 2. Set non-blocking in UDP socket to use tokio UdpSocket * Address block_on issues in unixpipe and low-latency transport * Fix bug * Use deref to call ZRuntime and tidy up unused imports * Try with interval * Use TX runtime only * Ignore files starting with underscore * Use all ZRuntimes * Fix the busy loop in unixpipe * Unifty the ztimeout macro and remove the zasync_executor_init * Port zenoh/tests to tokio * Fix the nested block_on issue * Remove trailing whitespaces * Fix the tests in io/zenoh-transport * Remove async-std in zenoh/zenoh * Remove async-std in zenohd * WIP on tests of I/O * Pass all tests in io and zenoh * cargo fmt * remove unused dashmap * Finish the porting on examples * Make from_config async again * Use tokio::net::lookup_host to convert the address to socket addresses asynchronously * Porting on zenoh-link-ws * Cargo fmt * Remove commented lines * Tokio porting on serial * Tidy up * Add a throughput quick test script * Tokio porting on udp * Update Cargo.lock * Correct the comment * Correctly return the result * Porting on zenoh-link-unixsocket_stream * Correct the feature flag of tokio-util * Port link-tls to tokio and bump workspace rustls to 0.22.2 * Port link-quic to tokio * Turn get_listeners and get_locators into async calls. Use AsyncMutex and AsyncRwLock explicitly * Tidy up ZRuntimes * Tokio porting on zenoh-plugin-storage-manager * Refine ZRuntime * Enable the std feature of zenoh-result to convert std::io::Result * Tokio porting on zenoh-sync * tokio porting on object_pool * tokio porting on plugins * tokio porting on zenoh-ext * Upgrade a few dependencies * Tokio porting on doc * Add the missing io trait * Skip those troublesome tests * Remove the unneeded ASYNC_STD_THREAD_COUNT * Cargo fmt * Use the environmental variable to configure ZRuntime * Cargo fmt * Tidy up tokio features * Add the std feature used in zenoh-result * Fix matching_status_* failures * Speed up routing test and work around the failure of three_node_combination test * Halve the chunk size * Cargo clippy * Prevent the race condition * Turn three_node_combination into a sequential test * Introduce CancellationToken to terminate anyway * It turns out to be the large size payload fail the test. * Fix deadlock by reintroducing five ZRuntimes * Revert the tokio-porting on plugins since it cause a deadlock in routing test * Adjust the ZRuntime properly to fix the deadlock in routing test * Align plugins to the main as we would skip them in tokio porting * Apply Luca's patch of object_pool to make it sync only * Fix bug in matching * Cargo clippy * Add back TimedEvent to be compatible with plugins * Fix tests * Quickfix for plugins * Rename token name * Remove the unused Signal * Introduce TaskTracker and CancellationToken into zenoh-link-tcp * Let's use ZRuntime::Reception to handle accept_task * Refine the testing scripts * Let CI decide the minimal number of ZRuntime threads * Make it easier to pass the test * Revert "Let CI decide the minimal number of ZRuntime threads" since this fail This reverts commit de5c757. * Cargo fmt & clippy * Make cargo nextest more resilient * Cargo fmt & clippy & doc test * Add the missing Windows snippet * Fix after rebase * Remove the unneeded async-std * Rename ZRuntime::Reception to ZRuntime::Acceptor * Refine the PR * Use tokio::time and slow down the message rate to 500 Hz in the downsampler test * Fix tokio::time and use tokio socket bind_device * Fix cargo clippy * Cargo fmt * Add the warning of 2ms resolution issue of tokio * Address the unused variables * Correct the error types * Address the platform compiling issue * Use warning while binding the interface * Fix cargo clippy * Use JoinHandle to manage the listening task * Refine the TODO comments --------- Co-authored-by: Luca Cominardi <luca.cominardi@gmail.com> * Fix the missing runtime issues in the bindings' tests (#808) * Put the spawning task on ZRuntime::TX * Fix the bug of an async runtime demanded by UdpSocket::from_std * Fix the permission denied while binding to None interface on some platforms (#826) * Bugfix --------- Co-authored-by: Luca Cominardi <luca.cominardi@gmail.com>
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Upgrade QUIC link to rustls 0.22.2. Currently it's blocked by Update rustls to 0.23 and ring to 0.17 quinn-rs/quinn#1715Maintain an older version of rustls in QUIC link. Upgrading the TLS link only.ZRuntime