Chunking tests across processes

[chrislloyd]

Do you want to request a feature or report a bug?
Feature

What is the current behavior?
While tests are executed in parallel across cores on a single machine, they can't be batched and executed across multiple processes or machines.

For large codebases with lots of tests, would could achieve better parallelization. Many CI environments provide environment variables around parallelization ($x of $n). For instance, check out Buildkite's docs on parallelizing builds.

Would this be something that Jest would consider supporting? Has it been attempted in the past? Are there any issues to take into consideration before attempting a PR?

[cpojer]

You can use -w to create as many processes as you'd like but given that most tests are CPU bound (unless they go to the network), this may actually be harmful for performance. We don't have a way to parallelize across machines but I would absolutely support a PR for this kind of stuff as it has the potential for massive impact at FB. Do you have a plan to build this?

[cpojer]

Not planning on changing this at the moment.

[apiv]

@chrislloyd @cpojer I know this ticket is quite old, but wanted to share this article with anyone who would like a workable (albeit hacky) way to split Jest across multiple CI containers:

https://labs.contactually.com/parallelizing-jest-across-multiple-ci-containers-551e7d3e4cb0

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.