Add backend timeout test for entity-bytes from IPIP-402

[lidel]

#56 introduced request-response tests for IPIP-402, including entity-bytes for range requests that return a minimal subset of blocks for passed byte range:

Request().
Path("/ipfs/{{cid}}", subdirWithMixedBlockFiles.MustGetCidWithCodec(0x70, "subdir", "multiblock.txt")).
Query("format", "car").
Query("dag-scope", "entity").
Query("entity-bytes", "512:1023"),

Tests included in #56 will confirm the response includes the correct subset of blocks for the file.

Extra mile for testing entity-bytes

Because fixture includes all blocks available for multiblock.txt file, the compliance test will not detect when a backend is broken and downloads the entire thing from the beginning (0:1023) just to return a slice of that (512:1023).

Such implementation will produce a valid response for cached data, but cache misses will be excruciatingly slow, and usually timeout.
If the file is 4GiB video and the client requested 1MiB in the middle, nothing will happen for a long long time, because the backend will be busy retrieving 2GiB of data just to return the last 1MiB.

I believe conformance suite should be testing this, because if our internal teams ended up with this footgun in Rhea/Saturn (brainstorming fix in filecoin-saturn/L1-node#415), we most likely will see more of this in the future.

How to test this

We should have additional test with partially-resolvable-1GiB-multiblock.bin, where only root block + child block responsible for requested range are present in the CAR fixture.

This way, the test will hang/timeout when an incorrectly implemented backend tries to fetch blocks that are not related to the entity-bytes request.

[lidel]

As for logistics, we could write custom go code, or it is possible to do it semi-manually with two Kubo repos: create big file from /dev/urandom, then export individual blocks via ipfs block get (from node being offline) and then on second node import them via ipfs block put and create dag-cbor envelope document that links to these CIDs (json → ipfs dag put), and then do ipfs dag export of the DAG of the envelope → produced .car fixture will only have 2 blocks, and the rest of the original DAG won't ever be available.

[laurentsenta]

@lidel I took the opportunity to learn about IPLD and car files,

Made a PR in ipld/go-car#457
Which enabled: #85

This is the code I use to generate the car files:

dd if=/dev/urandom of="file-3k-and-3-blocks.bin" bs=1024 count=3
CID=$(ipfs add ./file-3k-and-3-blocks.bin --chunker=size-1024 -q)
ipfs dag export $CID > file-3k-and-3-blocks.car
REMOVE_BLOCK=$(ipfs dag get $CID | jq '.Links[1].Hash["/"]' -r)
echo $REMOVE_BLOCK | car filter --version 1 --inverse ./file-3k-and-3-blocks.car ./file-3k-and-3-blocks-missing-block.car 

Let me know if that works for you!

[lidel]

Thank you @laurentsenta, this ability to filter CAR contents per CID is great addition to our CAR tooling ❤️