Cache tolerance for old archives

[arcanis]

What's the problem this PR addresses?

Each time we make a change to the libzip or zlib implementation, all generated archives typically see the compression output changed, with no notable changes on the actual content of the files. When the local cache is used this is reflected by git marking all cache files as changed, and requiring to push them anew. This is a very heavy process, especially since the compression prevents git from efficiently diffing the files. The situation is a little better with compression=0 (the new default), but not all projects can use it right away.

How did you fix it?

This diff makes Yarn tolerant to archives generated by older versions, as long as they have been generated by at least the cache version N (called the "checkpoint version"). The checksum is still validated.

This logic isn't enabled by default, by fear of adverse interactions with particular setups (in particular --check-cache, which is currently recommended for open-source projects using zero-installs; probably not many, but better to be careful so close from the next major release).

Checklist

• I have read the Contributing Guide.

• I have set the packages that need to be released for my changes to be effective.

• I will check that all automated PR checks pass before the PR gets reviewed.

[merceyz]

This logic is only enabled for projects using the local cache

This will break use cases where, for example, you use the global cache on the CI and the local cache on your machine.
Locally installs will pass while on the CI it will throw.

[arcanis]

I thought about this, and the only solution I could come up with was to introduce a new setting, cacheMigrationMode, which selects whether Yarn accepts old files or not. In the case you specify, you'd either set the CI value to required-only or your local value to always, and it should work.

This setting also allows us to let Yarn reuse files from different compression levels, so it has value in multiple use cases.

[merceyz]

This breaks yarn --check-cache as the new Yarn version with a different cache key can't reproduce the cache entry of the previous Yarn version.

[arcanis]

After some discussion I agree enabling this by default is too risky so close from the major release, due to potential issues with setups people may already have (in particular, the story with --check-cache isn't clear).

I turned back the default to always in every situation, but projects can still opt-in to required-only depending on their needs.