cmd/utils: grant snapshot cache to trie if disabled #21416
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
holiman
approved these changes
Aug 6, 2020
holiman
reviewed
Aug 6, 2020
| TrieCleanCacheJournal: "triecache", | ||
| TrieCleanCacheRejournal: 60 * time.Minute, | ||
| TrieDirtyCache: 256, | ||
| TrieTimeout: 60 * time.Minute, | ||
| SnapshotCache: 256, | ||
| SnapshotCache: 102, |
There was a problem hiding this comment.
102 looks very specific, I assume it's well backed by scientific research to pinpoint that it's better than 100
This was referenced Feb 24, 2021
enriquefynn
pushed a commit
to enriquefynn/go-ethereum
that referenced
this pull request
Mar 10, 2021
* cmd/utils: grant snapshot cache to trie if disabled * eth: fix up default non-mainnet cache distribution
tony-ricciardi
pushed a commit
to tony-ricciardi/go-ethereum
that referenced
this pull request
Jan 20, 2022
Cherry pick bug fixes from upstream for snapshots, which will enable higher transaction throughput. It also enables snapshots by default (which is one of the commits pulled from upstream). Upstream commits included: 68754f3 cmd/utils: grant snapshot cache to trie if disabled (ethereum#21416) 3ee91b9 core/state/snapshot: reduce disk layer depth during generation a15d71a core/state/snapshot: stop generator if it hits missing trie nodes (ethereum#21649) 43c278c core/state: disable snapshot iteration if it's not fully constructed (ethereum#21682) b63e3c3 core: improve snapshot journal recovery (ethereum#21594) e640267 core/state/snapshot: fix journal recovery from generating old journal (ethereum#21775) 7b7b327 core/state/snapshot: update generator marker in sync with flushes 167ff56 core/state/snapshot: gethring -> gathering typo (ethereum#22104) d2e1b17 snapshot, trie: fixed typos, mostly in snapshot pkg (ethereum#22133) c4deebb core/state/snapshot: add generation logs to storage too 5e9f5ca core/state/snapshot: write snapshot generator in batch (ethereum#22163) 18145ad core/state: maintain one more diff layer (ethereum#21730) 04a7226 snapshot: merge loops for better performance (ethereum#22160) 994cdc6 cmd/utils: enable snapshots by default 9ec3329 core/state/snapshot: ensure Cap retains a min number of layers 52e5c38 core/state: copy the snap when copying the state (ethereum#22340) a31f6d5 core/state/snapshot: fix panic on missing parent 61ff3e8 core/state/snapshot, ethdb: track deletions more accurately (ethereum#22582) c79fc20 core/state/snapshot: fix data race in diff layer (ethereum#22540) Other changes Commit f9b5530 (not from upstream) fixes an incorrect default DatabaseCache value due to an earlier bad merge. Tested Automated tests Testing on a private testnet Backwards compatibility Enabling snapshots by default is a breaking change in terms of the CLI flags, but will not cause backwards incompatibility between the node and other nodes. Co-authored-by: Péter Szilágyi <peterke@gmail.com> Co-authored-by: gary rong <garyrong0905@gmail.com> Co-authored-by: Melvin Junhee Woo <melvin.woo@groundx.xyz> Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Edgar Aroutiounian <edgar.factorial@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When we implemented the snapshotter, we distributed the
--cachememory allowance as:Before merging the snapshotter however, we hid it behind an opt-in flag
--snapshot. We did not however implement this opt-in mechanism in the cache distribution code, so if snapshots are not enabled, we ran with 90% allowance only. This PR fixes it by granting the snapshot cache back to the trie clean cache if snapshots are disabled.Fun fact, this fault only happens on mainnet or with custom
--cachesettings, because the defaults are calculated differently. Which were also "borked" as they followed an older hack where the snapshot cache was piled on top as 25% extra. This PR also fixes that for testnets by using 15+10 split instead of 25+25 (out of 25).