Improve IPNS publish time #7
Conversation
- Time measuring was fixed to actually work - Only 5 out of 20 peers are put to synchronously - No follow up when querying peers for GetClosestPeers - for PutValue only That last one was recommended by Adin and made the biggest difference, eliminating the majority of the publishing time..
This reverts commit 47dc901.
| // name publish operations | ||
| // Only publish to 5 out of 20 peers synchronously, the rest async | ||
|
|
||
| syncN := 5 |
There was a problem hiding this comment.
Do we even need the 5 peers to be sync? Is it so we can catch some errors?
Would it be possible to have all async requests and just wait for the first 5 responses?
There was a problem hiding this comment.
They could all be async, since we have pubsub. I thought it made sense to at least send to some peers, so that when the API request completes you know it's gone out to the DHT to at least some degree. Idk.
Would it be possible to have all async requests and just wait for the first 5 responses?
As I understand it, the only response in this scenario is just an acknowledgement that the publish message was received. It's possible there is no acknowledgement at all, but I think there is. So by synchronously sending the publish message to 5 peers, we're doing that already.
There was a problem hiding this comment.
As I understand it, the only response in this scenario is just an acknowledgement that the publish message was received. It's possible there is no acknowledgement at all, but I think there is. So by synchronously sending the publish message to 5 peers, we're doing that already.
What I was getting at is that doing it synchronously means that it's happening serially and we need to wait for each response to come in. But what we could do instead is do it in parallel and can get the responses out of order so that we can speed things up. E.g., each request happens in its own goroutine which writes to a channel on finish, then we listen to the first five responses and ignore the rest.
Is an individual synchronous request-response cycle fast enough that such an optimization wouldn't yield a big speedup?
There was a problem hiding this comment.
Ok so that is already happening. The message to each of the 5 peers is happening in its own goroutine. The "synchronous" part was that we wait for all 5 to complete before returning, but maybe that's a misnomer.
See commit messages for info. Maybe browse by commit because entire modules are brought in and are mostly unmodified.