Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/state/snapshot: reduce disk layer depth during generation #21477

Merged
merged 1 commit into from
Aug 24, 2020

Conversation

karalabe
Copy link
Member

The snapshotter operates on state (roots) and is oblivious to blocks. This is needed because there's really no notion of a block inside the state snapshot or the statedb and introducing them would make things extremely messy. This however creates a weird corner case:

  • The state GC operates on blocks and dereferences state that is 128 blocks old.
  • The snapshotter operates on states and flattens state that is 128 depths old.

If two blocks have the same state root (Clique empty block), these two 128 concepts will go out of sync. The snapshotter will still maintain 128 unique states, but that will correspond to more than 128 blocks. Essentially the snapshotter's disk layer will reference state that does not exist in trie form any more.

This is completely fine in normal mode of operation. The hiccup is during snapshot construction, because there it can happen that the trie based on which we are building the snapshot gets deleted. In that case in the existing code we need to wait for 128 non-empty blocks to shift the discrepancy out and resume generation. This is kind of bad on networks with limited tx throughput.

This PR makes a tiny hack so that during generation (and only then), the snapshot depth is kept at 8 instead of 128. This ensures that as long at there are at least 8 blocks with txs in them in a 128 block range, the snapshotter will be able to generate itself. The reason I picked 8 instead of 1 (which would work for a completely idle network too) is that the snapshotter's persistent layer can't reorg, so we need something that won't ever get reorged (during generation, or at least super improbable).

@karalabe karalabe added this to the 1.9.20 milestone Aug 24, 2020
@karalabe karalabe merged commit bdde616 into ethereum:master Aug 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants