Skip to content

engine/blobs: move partial return support to mandatory engine_getBlobsV3 #674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
+220 −11
Open

engine/blobs: move partial return support to mandatory engine_getBlobsV3 #674

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5cf47d7
add engine_getBlobsV3 method and BlobAndProofV3 type alias.
raulk Jul 8, 2025
1859a6e
actually restore old wording for engine_getBlobsV2.
raulk Jul 10, 2025
36f2d3e
drop BlobAndProofV3 alias; use BlobAndProofV2 type for engine_getBlob…
raulk Jul 10, 2025
aac419d
address minor review comments.
raulk Jul 14, 2025
a0e5252
fix spec point 3 in engine_getBlobsV3.
raulk Jul 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 184 additions & 5 deletions src/engine/openrpc/methods/blob.yaml
View file
Open in desktop

Large diffs are not rendered by default.

42 changes: 36 additions & 6 deletions src/engine/osaka.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ This specification is based on and extends [Engine API - Prague](./prague.md) sp
- [Request](#request-1)
- [Response](#response-1)
- [Specification](#specification-1)
- [engine_getBlobsV3](#engine_getblobsv3)
- [Request](#request-2)
- [Response](#response-2)
- [Specification](#specification-2)
- [Update the methods of previous forks](#update-the-methods-of-previous-forks)
- [Cancun API](#cancun-api)
- [Prague API](#prague-api)
Expand Down Expand Up @@ -89,7 +93,7 @@ This method follows the same specification as [`engine_getPayloadV4`](./prague.m

### engine_getBlobsV2

Consensus layer clients **MAY** use this method to fetch blobs from the execution layer blob pool.
Consensus layer clients **MAY** use this method to fetch blobs from the execution layer blob pool, in a all-or-nothing fashion. For partial response support, refer to `engine_getBlobsV3`.

#### Request

Expand All @@ -105,15 +109,41 @@ Consensus layer clients **MAY** use this method to fetch blobs from the executio

#### Specification

Refer to the specification for [`engine_getBlobsV1`](./cancun.md#engine_getblobsv1) with changes of the following:

1. Given an array of blob versioned hashes client software **MUST** respond with an array of `BlobAndProofV2` objects with matching versioned hashes, respecting the order of versioned hashes in the input array.
1. Given an array of blob versioned hashes, if client software has every one of the requested blobs, it **MUST** return an array of `BlobAndProofV2` objects whose order exactly matches the input array. For instance, if the request is `[A_versioned_hash, B_versioned_hash, C_versioned_hash]` and client software has `A`, `B` and `C` available, the response **MUST** be `[A, B, C]`.
2. If one or more of the requested blobs are unavailable, the client **MUST** return either `null` or an array of the same length and order, inserting `null` only at the positions of the missing blobs. For instance, if the request is `[A_versioned_hash, B_versioned_hash, C_versioned_hash]` and client software has data for blobs `A` and `C`, but doesn't have data for `B`, the response **MUST** be either `null` or `[A, null, C]`.
2. Client software **MUST** return `null` in case of any missing or older version blobs. For instance,
1. if the request is `[A_versioned_hash, B_versioned_hash, C_versioned_hash]` and client software has data for blobs `A` and `C`, but doesn't have data for `B`, the response **MUST** be `null`.
2. if the request is `[A_versioned_hash_for_blob_with_blob_proof]`, the response **MUST** be `null` as well.
3. Client software **MUST** support request sizes of at least 128 blob versioned hashes. The client **MUST** return `-38004: Too large request` error if the number of requested blobs is too large.
4. Client software **MUST** return `null` if syncing or otherwise unable to serve blob pool data.
4. Client software **MUST** return `null` if syncing or otherwise unable to generally serve blob pool data.
5. Callers **MUST** consider that execution layer clients may prune old blobs from their pool, and will respond with `null` if a blob has been pruned.

### engine_getBlobsV3

Consensus layer clients **MAY** use this method to fetch blobs from the execution layer blob pool, with support for partial responses. For an all-or-nothing query style, refer to `engine_getBlobsV2`.

#### Request

* method: `engine_getBlobsV3`
* params:
1. `Array of DATA`, 32 Bytes - Array of blob versioned hashes.
* timeout: 1s

#### Response

* result: `Array of BlobAndProofV2` - Array of [`BlobAndProofV2`](#BlobAndProofV2), inserting `null` only at the positions of the missing blobs, or a `null` literal in the designated cases specified below.
* error: code and message set in case an error occurs during processing of the request.

#### Specification

> To assist the reader, we highlight differences against `engine_getBlobsV2` using italic.

1. Given an array of blob versioned hashes client software **MUST** respond with an array of `BlobAndProofV2` objects with matching versioned hashes, respecting the order of versioned hashes in the input array.
2. Given an array of blob versioned hashes, if client software has every one of the requested blobs, it **MUST** return an array of _`BlobAndProofV2`_ objects whose order exactly matches the input array. For instance, if the request is `[A_versioned_hash, B_versioned_hash, C_versioned_hash]` and client software has `A`, `B` and `C` available, the response **MUST** be `[A, B, C]`.
3. If one or more of the requested blobs are unavailable, _the client **MUST** return an array of the same length and order, inserting `null` only at the positions of the missing blobs._ For instance, if the request is `[A_versioned_hash, B_versioned_hash, C_versioned_hash]` and client software has data for blobs `A` and `C`, but doesn't have data for `B`, _the response **MUST** be `[A, null, C]`. If all blobs are missing, the client software must return an array of matching length, filled with `null` at all positions._
4. Client software **MUST** support request sizes of at least 128 blob versioned hashes. The client **MUST** return `-38004: Too large request` error if the number of requested blobs is too large.
5. Client software **MUST** return `null` if syncing or otherwise unable to generally serve blob pool data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows an EL client to still return null as a response. From the CL's perspective V3 is identical to main's V2.

Copy link
Member Author

@raulk raulk Jul 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

V3 guarantees that partial responses will be served (unless no responses can be served at all, and this is not due to an internal error, which is what the null literal case covers here). Main's V2 does not make any such behavioural guarantee.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

V3 changes the EL behavior relative to V2, no argument there.

From the CL's perspective V3 is identical to main's V2.

This is my point. The CL has to handle receiving null or a list of partial responses, just as it does with V2. This is not a point against the PR, just a note.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a deliberate choice. V1 would return a null filled array, but it’s clearer semantics to return a single null literal that translates to an Option or pointer type in Rust and Go, given this outcome affects the whole request anyway. IMO it’s even cleaner to return an error, but I assumed there was a reason V1 didn’t from the get-go.

6. Callers **MUST** consider that execution layer clients may prune old blobs from their pool, and will respond with `null` at the corresponding position if a blob has been pruned.

### Update the methods of previous forks

#### Cancun API
Expand Down
Loading