test: jest improvements

[paul-soporan]

What's the problem this PR addresses?

This PR gradually upgrades Jest, enables some new features, and tries to improve the performance of running the test suites.

It's a draft because I want to gradually see how each improvement affects things (e.g. the speedup brought by upgrading to Jest 27, then the one brought by upgrading to Jest 28, then the one by enabling shards and so on).

Closes #4676.

How did you fix it?

• Upgrade to Jest 27
• Upgrade to Jest 28
• Don't specify modern timers since they're the new default
• Fix PnP tests on Windows (some paths didn't get converted)
• Update all Babel dependencies
• Use shards

Abandoned ideas:

• Caching the Jest cache, the difference is only around 30 seconds per workflow; IMO it isn't worth it due to the extra complexity
• Transpiling the test files using ESBuild (jest-esbuild), takes around 29m on ubuntu instead of 14m
• Using test.concurrent: It's very buggy, doesn't properly support snapshots, and our tests are currently written in a way that assumes sequential execution (setPackageWhitelist and startRegistryRecording use non-atomic global mutable state). In any case, test.concurrent doesn't seem to help that much on CI, since we already use --maxWorkers=100%, and each run call spawns a new process on a different thread. Because of this, test.concurrent is much more useful when running a single test file at a time.

Checklist

• I have read the Contributing Guide.

• I have set the packages that need to be released for my changes to be effective.

• I will check that all automated PR checks pass before the PR gets reviewed.

[merceyz]

Since this causes some of the Babel dependencies to get updated, could you update all of them?
There are some performance improvements that might help.¹²

yarn up "@babel/*"
yarn up "@babel/*" -R
yarn dedupe

Footnotes

1. https://github.com/babel/babel/blob/b58e35b6aa30a4c58da3147ae4a1fb5cef2073c9/CHANGELOG.md#running_woman-performance-2 ↩

2. https://github.com/babel/babel/blob/b58e35b6aa30a4c58da3147ae4a1fb5cef2073c9/CHANGELOG.md#running_woman-performance ↩

[V-for-Vasili]

Hello! Are there any plans to go forward with this? Specifically interested in Jest upgrade, as version 26.2.0 addresses significant issue with Posix find: jestjs/jest#10263

[paul-soporan]

Yep, I'll finish this PR today, sorry for the delay!

[paul-soporan]

Re: #4650 (comment)

Upgraded the Babel deps, not sure whether it actually improves perf since the results are within the margin of error (both positive and negative).

[paul-soporan]

I ended up using 2 shards for the ubuntu tests and 3 shards for the windows and macos ones since they're much slower. With these changes, all tests finish in 10 minutes at most.

@arcanis I think you have to update the protected branch settings to take into account the new check names since the old checks are still reported as and 5 expected checks .

[merceyz]

Excellent! 🎉