Handle relationship between boxo and go-car and go-merkledag #218
Comments
2023-03-21 Sync MeetingAttendeesGoalsMy primary goal in this conversation is how to best enable Bedrock to detach from the "boxo" effort, while also unblocking #196 AgendaItem 1: what to do with go-car
Item 2: what to do with go-merkledag
Item 3: any other disagreements that Rod wants to share? (Also encourage written thoughts to be shared publicly. They can be posted in this issue.) |
2023-03-21 meeting notesNote @BigLep is not great at listening well and taking notes. Below is some of what was covered... Rod: go-merkledag is not something that IPFS implementations should depend on. go-car Meta feedback Currently has low trust towards Kubo maintainers as...
Maybe can do a migration/breakage campaign but it can't be continual and it needs to be well thought through. Don't want to experience the repeated set of breakages like had with go-libp2p. Steve: dvd:
Steve: [There were other things, but the above was the gist I got written down.] defer: Action ItemsActions around go-car and go-merkledag were added to the top-level description. |
|
Clarifying ownershp of go-car and requiring CODEOWNER PR approval is happening in: |
Making clear the repo status per ipfs/boxo#218
|
Clarifying the status of go-merkledag and adding a COEOWNERS is happening in ipfs/go-merkledag#101 |
|
I plan to send this message to #bedrock in FIL Slack on 2023-03-28. Review / 👍 welcome. Hello Bedrock, I know various on the Bedrock team have been snagged by efforts from the @kubo-maintainers to consolidate various repos into "boxo" (formerly called go-libipfs). Here's a quick update on where we're at... Per conversations the week of 2023-03-20 in #218, go-car and go-merkledag no longer depend on Boxo. That dependency established earlier in 2023 has been reverted. The list of repos that will be getting copied into boxo (and their status) are listed here: #202 . If anything seems off, please share in #218 by EOD Friday 2023-03-31, as otherwise @kubo-maintainers will be adding "not maintained" notices to these repos and deprecated types starting 2023-04-03. boxo's next milestone is to be setup for more example-driven-development with a consolidated set of repos that can be versioned and refactored collectively can be found here. More info and tracking about that effort can be fond here: #196 Thanks and let us know if you have any questions, |
Making clear the repo status per ipfs/boxo#218 * Update README.md * Create CODEOWNERS * Fix Boxo link * fix: @rvagg only * doc: "maintained" but not actively developed, link to alternatives --------- Co-authored-by: Rod Vagg <rod@vagg.org>
|
I'm resolving this issue since
|
|
Can I make sure I’m understanding #202 correctly go-car and go-merkledag will remain in boxo, but also exist as independent forks (maintained by individuals from Bedrock)? That seems… like a worst of both worlds. Re go-car: why would IPFS stewards maintain a fork of code from the ipld organization? Why would two separate teams, both actually part of the Protocol Labs organization, maintain two separate copies of a repository? It just seems terrible. What expertise do IPFS stewards bring to a CAR library? How do they plan to develop it? Will they incorporate go-car changes (go-car is in active development and will remain so). How will the boxo version diverge from go-car? To me, the clear decision should be to simply have boxo use the external copies, rather than make two versions of the same software. The current decision seems like a concession to individuals, all at Protocol Labs, who have personal relationships with the stewards team, that's unmaintainable over the long term. It doesn’t address the underlying concern, which is the entire Boxo effort is creating friction not just for individuals but the whole ecosystem. I would rather we think about: what is best for the ecosystem? Because most people in the ecosystem don't have personal relationships. The fact that the end result notification is "send a message to a private slack channel" to me feels like further evidence that we're not having discussions in the open or thinking holistically about the ecosystem value of the entire Boxo project. If we're going with this, I sure hope it's temporary cause it seems untenable. Maybe stewards will show why go-car has to live inside of boxo and we decide we can remove go-car after all. If not, I hope we'll recognize that go-car is a seperate tool, maintained and developed most actively by other people, and there's no reason for it to live in boxo. |
|
I'm reopening as am already seeing some additional actions to take and will also work on responding to @hannahhoward |
|
Thanks for your message @hannah. Below are my thoughts and the actions I want to take as a result: A few things I want to make sure are clear and understood:
I’m not sure where the flag about not having discussions in the open is coming from. The message in the private FIL Slack #bedrock channel was an additional measure taken beyond all the other public steps that have been taken out of awareness that there is a large stream of notifications coming through. If there are other suggestions for working more in the open here, I certainly welcome hearing them. I think we have already hit “agree to disagree” on the value of the Boxo effort. At this piont, I think we need to get over the hump and we can evaluate after if the decision was right. Given the lack of shared cross-group certainty about this decision, our shifted emphasis has been to do it in a way that minimizes the blast-radius and impact on others in case it turns out to be a misstep. Better/consolidated justification for the Boxo work is in progress here. For the record, alternatives were attempted in the past around ipld/go-car:
🎬 I think these actions should be taken in light of this conversation. I’ll add them to the done criteria before closing this issue:
|
|
PR to better clarify boxo/ipld/car's relationship with ipld/go-car: #244 |
|
There has been some more discussion about a future with boxo depending on ipld/go-car if there's some refactoring in: #244 (comment) |
|
Per 2023-04-12 maintainer conversation, @Jorropo is going to respond here. |
|
I have posted a feedback about |
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
|
I updated the issue description to make clear that the done criteria is now to to get boxo/car removed. We can remove boxo/car once #291 is handled. |
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment)
* chore: migrate to boxo This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) * chore(deps): update to boxo v0.10.0 * chore(deps): update to tagged go-car --------- Co-authored-by: hannahhoward <hannah@hannahhoward.net>
* chore: migrate to boxo This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) * chore(deps): update to boxo v0.10.0 * chore(deps): update to tagged go-car --------- Co-authored-by: hannahhoward <hannah@hannahhoward.net>
* chore: migrate to boxo This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) * chore(deps): update to boxo v0.10.0 * chore(deps): update to tagged go-car --------- Co-authored-by: hannahhoward <hannah@hannahhoward.net>
This migrates everything except the `go-car` librairy: ipfs/boxo#218 (comment) I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups. This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649 The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions. Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent golang/go#58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type. Stubbing types were reverted ipfs/boxo#218 (comment) Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
|
This was closed given the work to replace boxolo/ipld/car by ipld/go-car in #400 . |
Current
Done criteria
boxo/car is removed. boxo will use ipld/go-car.
vvv OLD vvv
Done Criteria
Decision is made on the relation to be had between ipfs/boxo and ipld/go-car and ipfs/go-merkledag. This could mean having a dependency on ipfs/boxo (e.g., ipfs/go-merkledag#100 ) or boxo taking a fork of the relevant functionality it needs and maintaining it.
Why Important
The #196 effort (and particularly #206) can't be completed without decisions here, and that work is blocking other refactoring work like #176.
Resulting Actions
Below are the actions that are being taken as a result of the 2023-03-21 meeting:
✅ Item 1: Get go-car decoupled:
✅ Item 2: copy in go-merkledag to boxo
This will get handled as part of #202
A row will be added for go-merkledag.
No deprecated types will be added.
A CODOWNERS will be added.
ipfs/go-merkledag#101
Additional items that have come up since>
The text was updated successfully, but these errors were encountered: