Add autogenerated protos for `v1.0` #2

sergio-mena · 2023-12-07T10:26:26Z

This PR is the first to providing support for CometBFT v1.0.

The PR appears to be massive but the reason is the tons of newly auto-generated code it introduces. For this reason, please don't review it in one shot; rather use the following description of commits:

Commit 1: Ignore. It is the result of running the protobuf generation on my machine, which produces a few changes in the auto-generated code.
Commit 2: Review. First step: add the logic to handle both Tendermint and CometBFT protobufs. We might decide to deprecate the Tendermint part.
Commit 3: Ignore. The effect in auto-generated code of commit 2.
Commit 4: Review. Adapted protobuf generation tweaks to CombetBFT protos. Commit 2, just copy-pasted the Tendermint ones. This commit allows to see how I adapted them.
Commit 5: Ignore. The effect in auto-generated code of commit 4.
Commit 6: Review. As there are several levels of modules now, the preexisting mechanism to generate the modules was too rigid. This commit introduces and extended version
Commit 7: Ignore. The effect in auto-generated code of commit 6.
Commit 8: Review. Fixes needed in the manual code so that the new proto bufs builds correctly
Commit 9: Review. Changelog.

Note: even if this PR is likely to be ready for review, I decided to leave it a draft PR because we need to discuss whether we should be forking tendermint-rs into cometbft-rs and move this PR there.

Referenced an issue explaining the need for the change
Updated all relevant documentation in docs
Updated all code comments where relevant
Wrote tests
Added entry in .changelog/

sergio-mena · 2023-12-07T10:28:51Z

tools/proto-compiler/src/constants.rs

+    (".cometbft.libs.bits.BitArray", SERIALIZED),//TODO Revisit
+    (".cometbft.types.BlockIDFlag", PRIMITIVE_ENUM),
+    (".cometbft.types.Block", SERIALIZED),
+    (".cometbft.types.Data", SERIALIZED),
+    (".cometbft.types.EvidenceParams", SERIALIZED),
+    (".cometbft.types.Evidence.sum", SERIALIZED),
+    (".cometbft.types.Evidence.sum", TYPE_TAG),
+    (".cometbft.types.EvidenceList", SERIALIZED),
+    (".cometbft.types.DuplicateVoteEvidence", SERIALIZED),
+    (".cometbft.types.Vote", SERIALIZED),
+    (".cometbft.types.BlockID", SERIALIZED),
+    (".cometbft.types.PartSetHeader", SERIALIZED),
+    (".cometbft.types.LightClientAttackEvidence", SERIALIZED),
+    (
+        ".cometbft.types.LightClientAttackEvidence",
+        RENAME_ALL_PASCALCASE,
+    ),
+    (".cometbft.types.LightBlock", SERIALIZED),
+    (".cometbft.types.SignedHeader", SERIALIZED),
+    (".cometbft.types.Header", SERIALIZED),
+    (".cometbft.version.Consensus", SERIALIZED),
+    (".cometbft.types.Commit", SERIALIZED),
+    (".cometbft.types.CommitSig", SERIALIZED),
+    (".cometbft.types.ValidatorSet", SERIALIZED),
+    (".cometbft.crypto.PublicKey.sum", SERIALIZED),
+    (".cometbft.crypto.PublicKey.sum", TYPE_TAG),
+    (".cometbft.abci.ResponseInfo", SERIALIZED),
+    (".cometbft.types.CanonicalBlockID", SERIALIZED),
+    (".cometbft.types.CanonicalPartSetHeader", SERIALIZED),
+    (".cometbft.types.Validator", SERIALIZED),
+    (".cometbft.types.CanonicalVote", SERIALIZED),
+    (".cometbft.types.BlockMeta", SERIALIZED),
+    (".cometbft.types.TxProof", SERIALIZED),
+    (".cometbft.crypto.Proof", SERIALIZED),


Please ignore these changes: review them in Commit 4

sergio-mena · 2023-12-07T10:29:40Z

tools/proto-compiler/src/constants.rs

+        QUOTED_WITH_DEFAULT,
+    ),
+    (
+        ".cometbft.abci.ResponseInfo.last_block_height",
+        QUOTED_WITH_DEFAULT,
+    ),
+    (".cometbft.abci.ResponseInfo.last_block_app_hash", DEFAULT),
+    (
+        ".cometbft.abci.ResponseInfo.last_block_app_hash",
+        BYTES_SKIP_IF_EMPTY,
+    ),
+    (".cometbft.types.BlockID.hash", HEXSTRING),
+    (".cometbft.types.BlockID.part_set_header", RENAME_PARTS),
+    (
+        ".cometbft.types.PartSetHeader.total",
+        PART_SET_HEADER_TOTAL,
+    ),
+    (".cometbft.types.PartSetHeader.hash", HEXSTRING),
+    (".cometbft.types.Header.height", QUOTED),
+    (".cometbft.types.Header.time", OPTIONAL),
+    (".cometbft.types.Header.last_commit_hash", HEXSTRING),
+    (".cometbft.types.Header.data_hash", HEXSTRING),
+    (".cometbft.types.Header.validators_hash", HEXSTRING),
+    (".cometbft.types.Header.next_validators_hash", HEXSTRING),
+    (".cometbft.types.Header.consensus_hash", HEXSTRING),
+    (".cometbft.types.Header.app_hash", HEXSTRING),
+    (".cometbft.types.Header.last_results_hash", HEXSTRING),
+    (".cometbft.types.Header.evidence_hash", HEXSTRING),
+    (".cometbft.types.Header.proposer_address", HEXSTRING),
+    (".cometbft.types.Data.txs", NULLABLEVECARRAY),
+    (".cometbft.types.EvidenceList.evidence", NULLABLE),
+    (".cometbft.types.Commit.height", QUOTED),
+    (".cometbft.types.Commit.signatures", NULLABLE),
+    (".cometbft.types.CommitSig.validator_address", HEXSTRING),
+    (".cometbft.types.CommitSig.timestamp", OPTIONAL),
+    (".cometbft.types.CommitSig.signature", BASE64STRING),
+    (
+        ".cometbft.types.DuplicateVoteEvidence.total_voting_power",
+        RENAME_TOTAL_VOTING_POWER_QUOTED,
+    ),
+    (
+        ".cometbft.types.DuplicateVoteEvidence.validator_power",
+        RENAME_VALIDATOR_POWER_QUOTED,
+    ),
+    (
+        ".cometbft.types.DuplicateVoteEvidence.timestamp",
+        RENAME_TIMESTAMP,
+    ),
+    (
+        ".cometbft.types.LightClientAttackEvidence.common_height",
+        QUOTED,
+    ),
+    (
+        ".cometbft.types.LightClientAttackEvidence.total_voting_power",
+        QUOTED,
+    ),
+    (".cometbft.types.Vote.height", QUOTED),
+    (".cometbft.types.Vote.validator_address", HEXSTRING),
+    (".cometbft.types.Vote.signature", BASE64STRING),
+    (".cometbft.types.Vote.timestamp", OPTIONAL),
+    (".cometbft.types.Validator.address", HEXSTRING),
+    (
+        ".cometbft.types.Validator.voting_power",
+        ALIAS_POWER_QUOTED,
+    ), // https://github.com/tendermint/tendermint/issues/5549
+    (
+        ".cometbft.types.Validator.proposer_priority",
+        QUOTED_ALLOW_NULL,
+    ), // null occurs in some LightBlock data
+    (".cometbft.types.Validator.proposer_priority", DEFAULT), // Default is for /genesis deserialization
+    (
+        ".cometbft.types.ValidatorSet.total_voting_power",
+        QUOTED_WITH_DEFAULT,
+    ),
+    (
+        ".cometbft.types.ValidatorSet.total_voting_power",
+        SKIP_SERIALIZING,
+    ),
+    (".cometbft.types.BlockMeta.block_size", QUOTED),
+    (".cometbft.types.BlockMeta.num_txs", QUOTED),
+    (".cometbft.crypto.PublicKey.sum.ed25519", RENAME_EDPUBKEY),
+    (
+        ".cometbft.crypto.PublicKey.sum.secp256k1",
+        RENAME_SECPPUBKEY,
+    ),
+    (".cometbft.crypto.PublicKey.sum.sr25519", RENAME_SRPUBKEY),
+    (
+        ".cometbft.types.Evidence.sum.duplicate_vote_evidence",
+        RENAME_DUPLICATEVOTE,
+    ),
+    (
+        ".cometbft.types.Evidence.sum.light_client_attack_evidence",
+        RENAME_LIGHTCLIENTATTACK,
+    ),
+    (".cometbft.types.TxProof.data", BASE64STRING),
+    (".cometbft.types.TxProof.root_hash", HEXSTRING),
+    (".cometbft.crypto.Proof.index", QUOTED),
+    (".cometbft.crypto.Proof.total", QUOTED),
+    (".cometbft.crypto.Proof.aunts", VEC_BASE64STRING),
+    (".cometbft.crypto.Proof.leaf_hash", BASE64STRING),


Please ignore these changes: review them in Commit 4

mzabaluev

As we agreed on a call with @sergio-mena, I'm approving this to merge now and I can apply further changes in a separate PR.

sergio-mena added 10 commits November 17, 2023 15:11

Pre: auto changes

99e961d

Added cometbft v1_0 to protobuf generation

a935338

Auto generated protobuf code

95a2621

Adapted custom attributes for cometbft types

659ad9c

Auto generated protobuf code after adapting custom attributes

f60e631

Extended mod generation to support several sub-mods

0e70404

Auto generated protobuf code after extended mod generation

42a3555

Fix build of protos crate

dd1fcb8

Changelog

03e3aab

Addressed @mzabaluev's comments

f5ddeb1

sergio-mena commented Dec 7, 2023

View reviewed changes

sergio-mena mentioned this pull request Dec 7, 2023

Add autogenerated protos for v1.0 informalsystems/tendermint-rs#1381

Closed

5 tasks

sergio-mena added 5 commits December 15, 2023 15:32

Fix cometbft tags

f7adf75

Re-Fix build of protos crate

3bf1230

Fix main and constants

74f7085

Regenerated protobuf code

3769b58

cargo fmt

d7fd8ac

sergio-mena requested a review from mzabaluev December 15, 2023 16:37

sergio-mena mentioned this pull request Dec 15, 2023

cometbft-rs changes cometbft/cometbft#1834

Closed

5 tasks

Merge branch main into sergio/proto-renaming

805a57d

mzabaluev approved these changes Dec 18, 2023

View reviewed changes

sergio-mena merged commit 8f9df6f into main Dec 18, 2023
24 checks passed

sergio-mena deleted the sergio/proto-renaming branch December 18, 2023 13:17

mzabaluev mentioned this pull request Jan 15, 2024

Update cometbft-proto to protobufs in CometBFT v1.0 (pre-release) #7

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add autogenerated protos for `v1.0` #2

Add autogenerated protos for `v1.0` #2

sergio-mena commented Dec 7, 2023 •

edited

Loading

sergio-mena Dec 7, 2023

sergio-mena Dec 7, 2023

mzabaluev left a comment

Add autogenerated protos for v1.0 #2

Add autogenerated protos for v1.0 #2

Conversation

sergio-mena commented Dec 7, 2023 • edited Loading

sergio-mena Dec 7, 2023

Choose a reason for hiding this comment

sergio-mena Dec 7, 2023

Choose a reason for hiding this comment

mzabaluev left a comment

Choose a reason for hiding this comment

Add autogenerated protos for `v1.0` #2

Add autogenerated protos for `v1.0` #2

sergio-mena commented Dec 7, 2023 •

edited

Loading