chore(jest/circleci): setup test report for units

[Slashgear]

Description

• Generate test report on circle could help a lot contributors understand which test is broken.
• CircleCi has a great integration of report
  • CircleCi will attach test report to execution
    • number of test executed
    • Failed tests messages (exemple here)
  • In Insight circleci tab, maintainers will be able to see global stats on tests like
    • tests that fails the most
    • longest tests

💡 This could be done with cypress test or Eslint etc

Documentation

• Documentation on the integration of test reports on CircleCi

Related Issues

[pieh]

This looks great! I will get this in, but first we want to merge #23838 because there will be conflicts in circleCi config as both of those PRs touch on test_template and it will be easier to resolve them here (much less going on than the other PR)

[Slashgear]

@pieh We could consider adding this feature for other tests.
It is way simpler to see what the PR break, with report.

• cypress tests
• integration test

[pieh]

@pieh We could consider adding this feature for other tests.
It is way simpler to see what the PR break, with report.

• cypress tests
• integration test

Let's see how it goes with just jest unit tests for a week or so - if they prove to be trully valuable - we can enable those for other kinds of tests (I assume other kind of tests might need more involved setup as it's not as straight forward?)

[pieh]

Thanks! Let's try it out

[pieh]

Hey @Slashgear - we found some weirdness with Test summary:

For example in here - https://circleci.com/gh/gatsbyjs/gatsby/407019?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link there is failing thing - but it's not in assertions, it appears to be in setup - but it doesn't show in Test summary and also it doesn't log to "terminal output". Luckily we didn't setup test summary for windows tests ( https://circleci.com/gh/gatsbyjs/gatsby/407014?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link ) so we were able to lookup problem there - there is this in terminal output so we were able to pinpoint problem:

● Test suite failed to run

    TypeError: _opentracing.globalTracer.startSpan is not a function

      25 | const tracingContext = {
      26 |   graphqlTracing: false,
    > 27 |   parentSpan: globalTracer.startSpan(`test`),
         |                            ^
      28 | }
      29 | 
      30 | describe(`grapqhl-runner`, () => {

      at Object.startSpan (packages/gatsby/src/bootstrap/__tests__/create-graphql-runner.js:27:28)

Is it possible to setup it up to keep terminal output and not "pipe" everything to test summary? Which appears to not handle every case

[Slashgear]

@pieh Jest report will only generate report for failing tests executed.
If the problem is in setup it won't appear in generated report.

But I may have missed something and removed the default test reporter.
I take it back by add

--reporters="default"

I'm on it !

[pieh]

I guess specifying --reporters switch replaces the default list of reporters? So it would make sense that we explicitly need to add default one back if that's the case

[pieh]

https://jestjs.io/docs/en/configuration#reporters-arraymodulename--modulename-options

If custom reporters are specified, the default Jest reporters will be overridden. To keep default reporters, default can be passed as a module name.

[pieh]

Makes total sense as CLI switch as you wouldn't want have multiple reporters outputting to terminal - it's just with Test summary when it makes less sense :P

[Slashgear]

Solved it in ##24071