Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upgrade] Go-Ethereum release v1.9.21 #1211

Merged

Conversation

quorumbot
Copy link
Collaborator

@quorumbot quorumbot commented Jun 9, 2021

TODO

Plan & Analyse

  • Review the Release Notes
  • Review PRs in the section below

As you review, list extra changes and/or tests to be implemented to ensure compatibility with GoQuorum specific features.

Build & Test

  • Pull and checkout PR branch locally, then merge GoQuorum master into this branch
  • Resolve conflicts, taking into account the prior analysis
  • Implement required changes until lint passes
  • Implement required changes until all unit tests pass
  • Implement required changes until acceptance tests pass
  • Implement extra changes and/or tests
  • Verify any left TODOs in the code

Extra Changes & Tests

  • debug_traceCall : it's a new JSON-RPC API method that allows to call a contract with different arguments than debug_traceTransaction. This has been adapted so that it is compatible with private contract. Link to the source code
  • did not upgraded leveldb and protobuf because of our version of etcd. These upgrades cause some acceptance tests failures. Link to the commit

Left TODOs

  • We should upgrade the etcd we use v3.3.13-quorum197+incompatible so that it is compatible with protobuf 1.4.2. It will unblock the usage of a newer leveldb version. We could align the versions with original go-ethereum
    • Newer version of leveldb will fix a problem with released snapshots, have better performance, and a new configuration set. See PR

Go-Ethereum Release: Tavum (v1.9.21)

  • Version: v1.9.21
  • Published: 2020-09-09T08:25:11Z

Release notes

Geth v1.9.21 is a regular maintenance release, the highlights being the removal of whisper, better call tracing and multiple memory stability fixes during fast sync to both stabilize usage as well as to fix a memory leak that lead to crashers before.

  • Remove Whisper as promised a couple months ago (#21487, #21526, #21527)!
  • Minor user experience polishes around legacy Ledger derivation paths (#21517).
  • Implement arbitrary call tracing via debug_traceCall on top of arbitrary blocks (#21338).
  • Expose internal transaction revertals and revert reason in the call_tracer tracer (#21387).
  • Cap the number of in-memory trie nodes during fast sync, fix crasher memory leak (#21491).
  • Prepare the trie syncer for path-based operation to support the upcoming snap sync (#21504).
  • Limit the cached data in the downloader more aggressively to avoid memory fluctuations (#21366).
  • Fix the simulated chain to not allow changing block timestamps if transactions were included (#21334).
  • Fix an ABI parser issue around tuples (#21501).

For a full rundown of the changes please consult the Geth 1.9.21 release milestone


As with all our previous releases, you can find the:

Codebase changes assessment

Legend

File Stats: (A) Added, (M) Modified and (R) Removed

Line Stats: (A) Added and (R) Removed

Assessment:

  • ✅ No conflict expected
  • ⚠ Review required to assess changes
  • ‼️ Conflicts expected and review required

19 Pull Requests

🔍 Link Title File Stats
M/A/R
Packages changed
(files changed)
Line Stats
A/R
Top 5 Changed Files
(lines changed)
‼️ #21334 accounts/abi/bind/backends: Disallow AdjustTime for non-empty blocks 2/0/0
accounts/abi/bind/backends (2)
44/5
accounts/abi/bind/backends/simulated_test.go (82)
accounts/abi/bind/backends/simulated.go (16)
⚠️ #21490 eth: updated comments 3/0/0
eth (3)
3/3
eth/api.go (4)
eth/api_tracer.go (4)
eth/bloombits.go (4)
‼️ #21448 ``go.mod goleveldb latest update`` 2/0/0
go.mod (1)
go.sum (1)
54/17
#21491 core/state, eth, trie: stabilize memory use, fix memory leak 7/0/0
trie (4)
core/state (2)
eth/downloader (1)
58/29
trie/sync.go (120)
eth/downloader/downloader.go (22)
core/state/sync.go (12)
trie/committer.go (8)
core/state/statedb.go (4)
⚠️ #21387 eth/tracers: revert reason in call_tracer + error for failed internal calls 3/2/0
eth/tracers/internal/tracers (2)
eth/tracers/testdata (2)
eth/tracers (1)
172/15
eth/tracers/testdata/call_tracer_inner_instafail.json (144)
eth/tracers/testdata/call_tracer_revert_reason.json (128)
eth/tracers/tracers_test.go (48)
eth/tracers/internal/tracers/call_tracer.js (30)
eth/tracers/internal/tracers/assets.go (24)
#21497 rpc: fix issue with null JSON-RPC messages 3/0/0
rpc/testdata (2)
rpc (1)
16/3
rpc/json.go (26)
rpc/testdata/invalid-batch.js (6)
rpc/testdata/invalid-nonobj.js (6)
#21501 accounts/abi: fix a bug in getTypeSize method 2/0/0
accounts/abi (2)
37/1
accounts/abi/type_test.go (72)
accounts/abi/type.go (4)
#21503 internal: fix personal.sign() 1/0/0
console (1)
2/2
console/bridge.go (8)
#21455 "Downloader queue stats" is now provided once per minute 1/0/0
eth/downloader (1)
1/1
eth/downloader/queue.go (4)
‼️ #21432 go.mod: goja update 2/0/0
go.mod (1)
go.sum (1)
3/3
go.sum (8)
go.mod (4)
#21366 eth/downloader: change intial download size 5/0/0
eth/downloader (5)
26/24
eth/downloader/downloader_test.go (48)
eth/downloader/queue.go (28)
eth/downloader/queue_test.go (12)
eth/downloader/downloader.go (8)
eth/downloader/testchain_test.go (4)
#21504 core, eth, trie: prepare trie sync for path based operation 6/0/0
trie (4)
core/state (1)
eth/downloader (1)
480/105
eth/downloader/statesync.go (306)
trie/sync_test.go (298)
core/state/sync_test.go (254)
trie/trie.go (168)
trie/sync.go (132)
⚠️ #21338 eth: added trace_call to trace on top of arbitrary blocks 3/0/0
eth (2)
internal/web3ext (1)
54/8
eth/api_tracer.go (98)
eth/api.go (14)
internal/web3ext/web3ext.go (12)
‼️ #21514 accounts/abi/bid/backends: 2/0/0
accounts/abi/bind/backends (2)
76/76
accounts/abi/bind/backends/simulated.go (264)
accounts/abi/bind/backends/simulated_test.go (40)
#21487 whisper: remove whisper 6/0/24
whisper/whisperv6 (17)
mobile (3)
cmd/geth (3)
whisper/mailserver (2)
cmd/wnode (1)
tests/fuzzers/whisperv6 (1)
whisper/shhclient (1)
cmd/utils (1)
tests/fuzzers/whisperv6/corpus (1)
17/5826
whisper/whisperv6/filter_test.go (1672)
cmd/wnode/main.go (1546)
whisper/whisperv6/api.go (1186)
whisper/whisperv6/message_test.go (942)
whisper/whisperv6/message.go (710)
⚠️ #21517 accounts/usbwallet, signer/core: show accounts from ledger legacy derivation paths 2/0/0
accounts/usbwallet (1)
signer/core (1)
30/17
signer/core/api.go (66)
accounts/usbwallet/wallet.go (28)
⚠️ #21526 build: remove wnode from the list of packages binaries 1/0/0
build (1)
0/5
build/ci.go (10)
#21527 .github: remove whisper from CODEOWNERS 1/0/0
.github (1)
0/1
.github/CODEOWNERS (2)
⚠️ #21534 params: update CHTs for v1.9.21 release 1/0/0
params (1)
16/16
params/config.go (64)

72 Changed files

🔍 File Lines Changed Linked PR
whisper/whisperv6/whisper.go 2280
whisper/whisperv6/whisper_test.go 1856
whisper/whisperv6/filter_test.go 1672 #21487
cmd/wnode/main.go 1546 #21487
whisper/whisperv6/api.go 1186 #21487
whisper/whisperv6/message_test.go 942 #21487
whisper/whisperv6/message.go 710 #21487
whisper/whisperv6/envelope.go 560 #21487
whisper/whisperv6/peer.go 536 #21487
whisper/whisperv6/filter.go 524 #21487
whisper/mailserver/server_test.go 470 #21487
whisper/mailserver/mailserver.go 418 #21487
whisper/whisperv6/benchmarks_test.go 416 #21487
mobile/shhclient.go 390 #21487
whisper/shhclient/client.go 386 #21487
eth/downloader/statesync.go 306 #21504
trie/sync_test.go 298 #21504
‼️ accounts/abi/bind/backends/simulated.go 280 #21514
#21334
whisper/whisperv6/topic_test.go 268 #21487
core/state/sync_test.go 254 #21504
trie/sync.go 240 #21491
#21504
mobile/types.go 186 #21487
whisper/whisperv6/doc.go 184 #21487
whisper/whisperv6/envelope_test.go 182 #21487
tests/fuzzers/whisperv6/whisper-fuzzer.go 180 #21487
whisper/whisperv6/gen_newmessage_json.go 180 #21487
trie/trie.go 172 #21491
#21504
whisper/whisperv6/gen_message_json.go 168 #21487
eth/tracers/testdata/call_tracer_inner_instafail.json 144 #21387
whisper/whisperv6/gen_criteria_json.go 132 #21487
whisper/whisperv6/api_test.go 128 #21487
‼️ go.sum 128 #21432
#21448
eth/tracers/testdata/call_tracer_revert_reason.json 128 #21387
‼️ accounts/abi/bind/backends/simulated_test.go 122 #21514
#21334
whisper/whisperv6/topic.go 112 #21487
‼️ eth/api_tracer.go 102 #21490
#21338
accounts/abi/type_test.go 72 #21501
⚠️ signer/core/api.go 66 #21517
⚠️ params/config.go 64 #21534
whisper/whisperv6/config.go 62 #21487
‼️ cmd/utils/flags.go 54 #21487
‼️ cmd/geth/config.go 54 #21487
⚠️ eth/downloader/downloader_test.go 48 #21366
⚠️ eth/tracers/tracers_test.go 48 #21387
eth/downloader/queue.go 32 #21455
#21366
⚠️ eth/downloader/downloader.go 30 #21491
#21366
eth/tracers/internal/tracers/call_tracer.js 30 #21387
⚠️ accounts/usbwallet/wallet.go 28 #21517
‼️ go.mod 26 #21432
#21448
⚠️ rpc/json.go 26 #21497
eth/tracers/internal/tracers/assets.go 24 #21387
mobile/geth.go 20 #21487
‼️ eth/api.go 18 #21490
#21338
‼️ cmd/geth/consolecmd_test.go 16 #21487
core/state/sync.go 12 #21491
eth/downloader/queue_test.go 12 #21366
trie/secure_trie.go 12 #21504
⚠️ internal/web3ext/web3ext.go 12 #21338
⚠️ build/ci.go 10 #21526
trie/committer.go 8 #21491
console/bridge.go 8 #21503
rpc/testdata/invalid-nonobj.js 6 #21497
rpc/testdata/invalid-batch.js 6 #21497
⚠️ eth/bloombits.go 4 #21490
trie/trie_test.go 4 #21491
⚠️ cmd/geth/usage.go 4 #21487
⚠️ core/state/statedb.go 4 #21491
accounts/abi/type.go 4 #21501
eth/downloader/testchain_test.go 4 #21366
⚠️ params/version.go 4
.github/CODEOWNERS 2 #21527
tests/fuzzers/whisperv6/corpus/009c5adfa4fd685caef58e1ce932fa7fb209730a 0 #21487

fjl and others added 25 commits August 25, 2020 16:21
…(#21334)

* accounts/abi/bind/backends: Disallow timeshift for non-empty blocks

* accounts/abi/bind/backends: added tests for adjust time

* accounts/abi/bind/simulated: added comments, fixed test for AdjustTime

* accounts/abi/bind/backends: updated comment
* go.mod | goleveldb latest update

* go.mod update

* leveldb options

* go.mod: double check

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
core/state, eth, trie: stabilize memory use, fix memory leak
… calls (#21387)

* tests: add testdata of call tracer

* eth/tracers: return revert reason in call_tracer

* eth/tracers: regenerate assets

* eth/tracers: add error message even if no exec occurrs, fixes #21438

Co-authored-by: Martin Holst Swende <martin@swende.se>
* accounts/abi: fix a bug in getTypeSize method

e.g. for "Tuple[2]" type, the element of the array is a tuple type and the size of the tuple may not be 32.

* accounts/abi: add unit test of getTypeSize method
* "Downloader queue stats" is now a DEBUG information

I think this info is more a DEBUG related information then an INFO. If it must remains an INFO, maybe it can be slow down to one time every 5 minutes or so.

* Update queue.go

"Downloader queue stats" information is now provided once every minute instead of once every 10 seconds.
This changes how the downloader works, a little bit. Previously, when block sync started,
we immediately started filling up to 8192 blocks. Usually this is fine, blocks are small
in the early numbers. The threshold then is lowered as we measure the size of the blocks
that are filled.

However, if the node is shut down and restarts syncing while we're in a heavy segment,
that might be bad. This PR introduces a more conservative initial threshold of 2K blocks
instead.
core, eth, trie: prepare trie sync for path based operation
* eth: Added TraceTransactionPending

* eth: Implement Trace_Call, remove traceTxPending

* eth: debug_call -> debug_traceCall, recompute tx environment if pruned

* eth: fix nil panic

* eth: improve block retrieving logic in tracers

* internal/web3ext: add debug_traceCall to console
…nd anonymous variable assignments (#21514)

Co-authored-by: Osoro Bironga <osoro@doctaroo.com>
* whisper: remove whisper

* Update cmd/geth/config.go

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

* cmd/geth: warn on enabling whisper + remove more whisper deps

* mobile: remove all whisper references

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
…ivation paths (#21517)

* accounts/usbwallet, signer/core: un-hide accounts from ledger legacy derivation paths

* Update accounts/usbwallet/wallet.go

* Update signer/core/api.go

* Update signer/core/api.go
params: update CHTs for v1.9.21 release
@ricardolyn ricardolyn marked this pull request as ready for review June 15, 2021 09:52
Copy link
Contributor

@nmvalera nmvalera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -44,7 +44,6 @@ require (
github.com/golang/mock v1.4.3
github.com/golang/protobuf v1.3.4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a comment here to maintain the old version?

Copy link
Contributor

@baptiste-b-pegasys baptiste-b-pegasys Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean add comment in order to keep the geth's version?
I prefer to prepare another PR to solve the remaining TODOs, which is a revert of the last commit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@baptiste-b-pegasys can you create an issue to track this, so we are aware we need to put back the original version from geth?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the meantime, I can merge this.

@ricardolyn ricardolyn merged commit 94d12a9 into Consensys:master Jun 15, 2021
@ricardolyn ricardolyn deleted the upgrade/go-ethereum/v1.9.21-2021609142421 branch June 15, 2021 12:57
@ricardolyn ricardolyn added this to the v21.4.X (Future Patch) milestone Jun 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.