Extension Signing request endpoint #982

usmansaleem · 2024-03-20T07:27:32Z

PR Description

An extension sign endpoint /api/v1/eth2/ext/sign/:identifier which can be enabled with cli option --Xsigning-ext-enabled=true The cli option is associated with eth2 subcommand. For example ./web3signer eth2 --Xsigning-ext-enabled=true .... This extension covers some additional signing types and use-cases which are not covered by the remoting API specs and hence should be used at your own risk.

The format of request payload is:

{
  "type" : "PROOF_OF_VALIDATION",
  "platform" : "dappnode",
  "timestamp" : "1711338489397"
}

Only PROOF_OF_VALIDATION extension type is supported at the moment. Any other type would result in an error.

The signature is hex encoded string calculated as:

signature = BLS.sign(payload.getBytes()).asHexString()

For default accept type (*/*) and JSON accept type headers, the response is in following format:
Note:
Payload is returned in the response as Base64 encoded String.
Signature is returned as Hex encoded String

{
    "payload": "eyJ0eXBlIjoiUFJPT0ZfT0ZfVkFMSURBVElPTiIsInBsYXRmb3JtIjoiQVQiLCJ0aW1lc3RhbXAiOiIxNzE0NTI0NTQ0NDA3In0=",
    "signature": "0xb792ace1bae87b02457957bfa753065094e153855049074c880b943e7239d4277f4b2384d6aa19a3c03043b92bc8e90302ae48b9ac6b0a5e50796fc5e1d3993e9df6621ad6906e7a754147a9ccacdc9e7c18c1be355a647db19591904575bd30"
}

This would allow the users of this endpoint to verify the payload is signed by appropriate private key.

Fixed Issue(s)

Fixes #976

Documentation

I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

I thought about adding a changelog entry, and added one if I deemed necessary.

Testing

I thought about testing these changes in a realistic/non-local environment.

...h/pegasys/web3signer/core/service/http/handlers/signing/GenericSignForIdentifierHandler.java

core/src/main/java/tech/pegasys/web3signer/core/config/BaseConfig.java

core/src/main/java/tech/pegasys/web3signer/core/Runner.java

acceptance-tests/src/test/java/tech/pegasys/web3signer/dsl/signer/Signer.java

jframe · 2024-04-08T04:45:11Z

...in/java/tech/pegasys/web3signer/core/service/http/handlers/signing/SigningExtensionBody.java

+    @JsonProperty(value = "platform", required = true) String platform,
+    @JsonProperty(value = "timestamp", required = true) String timestamp,
+    @JsonProperty(value = "pubkey", required = true) String pubkey) {


These look like specific properties for a PROOF_OF_VALIDATION type rather than a generic signing type. Perhaps just rename SigningExtensionBody to ProofOfValidationBody?

jframe · 2024-04-08T04:51:54Z

...in/java/tech/pegasys/web3signer/core/service/http/handlers/signing/SigningExtensionBody.java

+  }
+
+  private static String headerBase64() {
+    return BaseEncoding.base64().encode("{\"alg\": \"BLS\", \"typ\": \"BLS_SIG\"}".getBytes(UTF_8));


Why does this need a header? This is done differently from how we sign everything else.

jframe · 2024-04-08T04:55:37Z

...asys/web3signer/core/service/http/handlers/signing/SigningExtensionForIdentifierHandler.java

+      routingContext.response().putHeader("Content-Type", JSON_UTF_8);
+      routingContext
+          .response()
+          .end(new JsonObject().put("data", dataToSign).put("signature", blsSigBase64).encode());


We don't need to include the original data, the sender already hash that. And this change response format to being specific for extension signing

…anager library

siladu

What do we now in terms of API docs for this?

I'm thinking it will be hard for a future user to understand how to use this API, e.g. what should they put for "platform"? Also, wondering if platform is strictly necessary in the request?

siladu · 2024-04-16T02:43:07Z

CHANGELOG.md

@@ -2,6 +2,9 @@

 ## Next Version

+### Features Added
+- Added endpoint `/api/v1/eth2/ext/sign/:identifier` which is enabled using cli option `--Xsigning-ext-enabled=true`. This endpoint allows signing of additional data not covered by the remoting API specs. [#982](https://github.com/Consensys/web3signer/pull/982)


Wonder if we should put this under a web3signer-specific namespace? e.g. /w3s/v1/eth2/sign, /w3s/v1/sign or/v1/eth2/w3s/sign.

There's precedent in teku for spec vs teku-specific https://consensys.github.io/teku/#tag/Teku and I think there's some merit for web3signer to follow that pattern.
Other CLs too: https://lighthouse-book.sigmaprime.io/api-lighthouse.html

siladu · 2024-04-16T02:52:01Z

core/src/main/java/tech/pegasys/web3signer/core/service/http/OpenApiOperationsId.java

@@ -13,6 +13,7 @@
 package tech.pegasys.web3signer.core.service.http;

 /** Operation IDs as defined in web3signer.yaml */
+@Deprecated(forRemoval = true)


why is this deprecated?

this class was used when we had vertx "openapi router". We have since removed that router and this interface is unused.

siladu · 2024-04-16T02:58:23Z

...n/java/tech/pegasys/web3signer/core/service/http/handlers/signing/ProofOfValidationBody.java

+public record ProofOfValidationBody(
+    @JsonProperty(value = "type", required = true) SigningExtensionType type,
+    @JsonProperty(value = "platform", required = true) String platform,
+    @JsonProperty(value = "timestamp", required = true) String timestamp) {}


Can this be timestamp typed? Makes sense to me to be the same as other timestamps, i.e. UInt64

Not sure platform should be types...what is platform actually used for?

I am thinking to use uint64 as well because that would allow numerical literals both quoted and unquoted.

Uses custom copy of global ObjectMapper with FAIL_ON_UNKNOWN_PROPERTIES enabled

...java/tech/pegasys/web3signer/core/service/http/handlers/signing/SigningExtensionHandler.java

-- Disable IPv6 in docker container for goss_wait test

usmansaleem added 5 commits March 20, 2024 10:48

feat!: Generic signing extension handler

a9e8973

cq: Remove unnecessary LOG variable

e4bf550

feat: Use base64 encoding of payload in response

91cfe43

tests - Add acceptance tests

03ced53

changelog

2e99ece

usmansaleem marked this pull request as ready for review March 21, 2024 02:13

usmansaleem requested review from siladu, jframe and gfukushima March 21, 2024 02:13

jframe reviewed Mar 22, 2024

View reviewed changes

usmansaleem added 2 commits March 25, 2024 13:32

fix: Use specific body for extension signing

b3d5707

test: Update Acceptance tests

c12c749

usmansaleem changed the title ~~Generic Sign endpoint~~ Extension Signing request endpoint Mar 25, 2024

usmansaleem added 2 commits March 27, 2024 06:12

Update cli option

12e5b40

Deleting Eth1 AT

423e042

usmansaleem marked this pull request as draft March 26, 2024 21:10

usmansaleem added 2 commits March 27, 2024 09:46

Refactor signing data to use header.payload.sig format

ee024bc

Refactor signing extension to eth2 mode. Fix acceptance test

8d08221

usmansaleem marked this pull request as ready for review March 27, 2024 02:28

usmansaleem requested a review from jframe March 27, 2024 02:28

usmansaleem added 6 commits March 28, 2024 09:01

spotless fix

87ea174

Merge remote-tracking branch 'upstream/master' into sign_extension

d413431

changelog

4cd29c6

changing json response format

9422720

Merge remote-tracking branch 'upstream/master' into sign_extension

bf3dada

Merge remote-tracking branch 'upstream/master' into sign_extension

a1fea94

jframe reviewed Apr 8, 2024

View reviewed changes

usmansaleem added 3 commits April 15, 2024 11:42

fix: Update bouncycastle libraries

5a2903f

fix: Update transitive dependency threetenbp and google cloud secretm…

f7cb63e

…anager library

build: assign dependency scan nvd api key from env variable

0c11f44

usmansaleem added 5 commits April 15, 2024 11:53

changelog

9aa7ecc

fix: Update guava and commons-logging libraries

11c1dd6

Merge branch 'vuln_scan_fix' into sign_extension

458975c

fix: Update response to return hex encoded signature

23984b2

Merge remote-tracking branch 'upstream/master' into sign_extension

7f06899

usmansaleem requested a review from jframe April 15, 2024 05:53

siladu reviewed Apr 16, 2024

View reviewed changes

usmansaleem added 4 commits May 1, 2024 10:21

Enforce strict fields parsing for SigningExtensionHandler

17692cf

Uses custom copy of global ObjectMapper with FAIL_ON_UNKNOWN_PROPERTIES enabled

Send Base64 encoded payload in the response

3c60ee6

Use appropriate type for tinestamp

e58d150

Merge remote-tracking branch 'upstream/master' into sign_extension

c4fe46f

jframe reviewed May 1, 2024

View reviewed changes

...java/tech/pegasys/web3signer/core/service/http/handlers/signing/SigningExtensionHandler.java Show resolved Hide resolved

usmansaleem added 2 commits May 3, 2024 07:57

Merge remote-tracking branch 'upstream/master' into sign_extension

6cbae83

fix: Always use application/json as Content-Type header

38a6db7

usmansaleem requested a review from jframe May 8, 2024 09:23

usmansaleem added 3 commits May 9, 2024 09:24

build - debug information in docker test.sh

a6eb923

build - fix docker goss test

acf0714

-- Disable IPv6 in docker container for goss_wait test

build - store docker dgoss test results

e11ae2b

jframe approved these changes May 9, 2024

View reviewed changes

usmansaleem merged commit 1c039cb into Consensys:master May 9, 2024
7 checks passed

usmansaleem deleted the sign_extension branch May 9, 2024 23:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extension Signing request endpoint #982

Extension Signing request endpoint #982

usmansaleem commented Mar 20, 2024 •

edited

Loading

jframe Apr 8, 2024

jframe Apr 8, 2024

jframe Apr 8, 2024

siladu left a comment

siladu Apr 16, 2024

siladu Apr 16, 2024

usmansaleem Apr 23, 2024

siladu Apr 16, 2024

usmansaleem Apr 23, 2024

usmansaleem May 1, 2024

Extension Signing request endpoint #982

Extension Signing request endpoint #982

Conversation

usmansaleem commented Mar 20, 2024 • edited Loading

PR Description

Fixed Issue(s)

Documentation

Changelog

Testing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

siladu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

usmansaleem commented Mar 20, 2024 •

edited

Loading