Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipldutil: simplify state synchronization, add docs #300

Merged
merged 1 commit into from
Dec 17, 2021
Merged

ipldutil: simplify state synchronization, add docs #300

merged 1 commit into from
Dec 17, 2021

Conversation

mvdan
Copy link
Contributor

@mvdan mvdan commented Dec 3, 2021

(see commit message)

Fixes #274.

@mvdan
Copy link
Contributor Author

mvdan commented Dec 3, 2021

I've stared at the code for a while, and I think understanding the TODO bits will help unblock me to document the rest of the code.

Copy link
Collaborator

@hannahhoward hannahhoward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks this is helpful! Will reach out about next steps

ipldutil/traverser.go Show resolved Hide resolved
ipldutil/traverser.go Outdated Show resolved Hide resolved
ipldutil/traverser.go Outdated Show resolved Hide resolved
ipldutil/traverser.go Outdated Show resolved Hide resolved
@rvagg
Copy link
Member

rvagg commented Dec 6, 2021

Is the intent to resolve the TODOs on this branch, or do them piecemeal in future updates? Other than that comment block that Hannah has provided some clarity on this lgtm since it doesn't look like it has material impact.

@mvdan mvdan changed the title ipldutil: first batch of refactors and mental TODOs ipldutil: simplify state synchronization, add docs Dec 9, 2021
@mvdan
Copy link
Contributor Author

mvdan commented Dec 9, 2021

@rvagg apologies for the late reply - the TODOs were open questions for Hannah, really. The intent was to have them during the early stages of code review so I could understand how the code worked.

@mvdan
Copy link
Contributor Author

mvdan commented Dec 9, 2021

@hannahhoward this should be ready for review and/or testing. Please pay extra attention to how the mutex is documented and used, in particular to the loader method.

@mvdan
Copy link
Contributor Author

mvdan commented Dec 15, 2021

Friendly bump that this is ready for review/merge :)

Copy link
Collaborator

@hannahhoward hannahhoward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about deadlocking on multiple calls to Advance/Error after the traversal is done. I mean generally this shouldn't happen anyway, but I think you want to release the lock unless you are actually sending a response over the nextResponse channel

ipldutil/traverser.go Show resolved Hide resolved
ipldutil/traverser.go Show resolved Hide resolved
ipldutil/traverser.go Show resolved Hide resolved
The two goroutines were synchronized via two channels,
awaitRequest to signal when StorageReadOpener starts,
and responses to provide it with a value to unblock and return.

Besides those two, there was also stateChan, to pass state around.

Keep the responses channel in place,
but replace awaitRequest and stateChan with a mutex and plain fields.
The added docs describe how they are used and why.

Note that we have to be  careful when locking and unlocking the mutex.
We can't unlock an unlocked mutex, nor leave the mutex locked.
Each call to Lock or Unlock comes with a reasoning.

parentCtx isn't used beyond the constructor, so remove the field.
The visitor and chooser defaults are fallbacks, so use them as such.

Use explicit struct keys for literals, making the code easier to follow.

Fixes #274.
@mvdan
Copy link
Contributor Author

mvdan commented Dec 17, 2021

Should be ready for a final review+merge I think :)

@mvdan
Copy link
Contributor Author

mvdan commented Dec 17, 2021

Seems like a flake on TestNetworkDisconnect. Does that ring a bell to either of you? I don't imagine my change is the cause.

@hannahhoward hannahhoward merged commit faf7a5c into ipfs:main Dec 17, 2021
@mvdan mvdan deleted the ipldutil-first branch January 10, 2022 15:14
marten-seemann pushed a commit that referenced this pull request Mar 2, 2023
* feat(deps): update graphsync to 2.0 branch

Provide the absolute minimum required to work with the latest version of
graphsync with the 2.0 protocl.

This does use a very unfortunate hack to deal with the change to
extension type -- it now deserializes/serializes from basicnode.Any
in order to get back to a []byte type

* style(mod): tidy

* refactor(message): create To/From IPLD funcs

* fix(impl): add one more int conversion

* feat(msg): ipld-prime messages, compat testing w/ cbor-gen

* fix(msg): replace message 1_1 with 1_1prime form

* chore(msg): rename struct fields, use renames for consistent encoding

* fix(msg): remove through-bytes roundtrip for conversions

* fix(msg): fixed upstream bindnode bug, added some temporary debugging

* fix(graphsync): compare Nodes with ipld.DeepEqual

* fix(deps): update ipld-prime with bindnode tuple fix

* fix(msg): remove debug prints, fix imports

* fix(cbor-gen): make go generate work again in old msgs

* chore(deps): update go-ipld-prime to new master with tuple fix

* feat(graphsync): remove peer id tracking

no longer need to utilize peer id to tract graphsync requests since RequestIDs are unique

* feat(deps): update to tagged go-graphsync

Co-authored-by: Rod Vagg <rod@vagg.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document ipldutil.Traverser
3 participants