codeintel: allow configuring max concurrency in uploads #1023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
efritz
approved these changes
Aug 22, 2023
Strum355
force-pushed
the
nsc/codeintel-upload-concurrency
branch
from
c31f212
to
6b72a07
Compare
Strum355
added a commit
to sourcegraph/sourcegraph-public-snapshot
that referenced
this pull request
Aug 28, 2023
Our HTTP endpoints have a [hard-coded timelimit of 75s](https://sourcegraph.com/search?q=context:global+repo:%5Egithub%5C.com/sourcegraph/sourcegraph%24+75+*+lang:Go&patternType=standard&sm=1&groupBy=path). On slower connections with multipart uploads, it is possible for the bandwidth congestion of the competing upload parts to result in the upload failing due to any part taking longer than 75s to upload. This PR adds a new toggleable to configure the number of parts to be uploaded concurrently, so reduce the congestion/competing between parts. ## Test plan Tested in combination with src-cli over at sourcegraph/src-cli#1023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same details as https://github.com/sourcegraph/sourcegraph/commit/de4d5471e3f61440cfe7837f56f25b6e778c5473
Test plan
Value is correctly parsed and set by printing the value with a full e2e run, concurrency tested by conc library itself