Document CI jobs

[gibfahn]

• Subsystem: doc, build

See nodejs/help#548, we should briefly document the jobs we have in CI, probably in the COLLABORATOR_GUIDE.

https://ci.nodejs.org/job/node-stress-single-test/

cc/ @vsemozhetbyt

[vsemozhetbyt]

@gibfahn I am not sure for now how to do this, need some experiments (the first attempt has failed, maybe I've configured something wrong). Maybe it would be better if anybody with more experience here documents this.

[joyeecheung]

Er, I am not sure if this is actually a good first contribution? (unless mentors are available)

[gibfahn]

@joyeecheung mentors are available 😄

I think what we need is a list of the ci jobs that collaborators (or any other contributor) need to know about. ci.nodejs.org is open to anyone to look at.

The jobs I can think of are:

• https://ci.nodejs.org/job/node-test-commit/ or https://ci.nodejs.org/job/node-test-pull-request/
  • probably already documented
• https://ci.nodejs.org/job/node-stress-single-test/
• https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/
  • possibly also documented

For each one we'd need a link and a one sentence description of what they are for. Just so that new collaborators get told this as part of Onboarding, instead of having to find out by chance later.

[vsemozhetbyt]

@gibfahn It seems the https://ci.nodejs.org/job/node-test-linter/ is also worth documenting for PRs concerning files that are not relevant for builds (like benchmarks).

[morrme]

I'm interested in helping with this, with support from a mentor.

[gibfahn]

@morrme Thanks!

So this info should be added to the CI Testing section of either CONTRIBUTING.md or the onboarding.md doc, depending on whether you think it'd be useful for people who aren't collaborators (for example, if you think this info would be useful for you as you read CONTRIBUTING.md?)

We probably want a sentence saying that our test jobs run from ci.nodejs.org, and then give links to some of the most commonly used jobs (i.e. those mentioned above).

A guide to the mechanics of raising a PR can be found in CONTRIBUTING.md as well.

If you need any help, just comment in here. A PR can also be raised for feedback before it's finished if you'd like.

[morrme]

@gibfahn Thanks! It is definitely important information for a collaborator and I personally find it interesting as a contributor as well.

So far, there are 5 tests listed.

I found a little information about node-stress_single-test from issue #3854 . Are there any other notes I can use to come up with the descriptions?

[gibfahn]

@morrme I think all you'd really need to say is that the job allows you to specify a set of tests to pass to tools/test.py, and that there's a repeat option to run a test or tests in a loop. A couple of examples might be helpful as well (cc/ @Trott, we were discussing this in a PR somewhere, can't remember where).

[Trott]

@morrme I think all you'd really need to say is that the job allows you to specify a set of tests to pass to tools/test.py, and that there's a repeat option to run a test or tests in a loop. A couple of examples might be helpful as well (cc/ @Trott, we were discussing this in a PR somewhere, can't remember where).

For an initial pass, the repeat info can be left off. We can add it later (or not). The main thing about node-stress-single-test is that it's designed to allow one to run a single test over and over on a specific platform to confirm that the test is reliable.

[morrme]

@gibfahn OK so for the ones you've marked as "possibly already documented", I cannot seem to find anything on these.

And for @vsemozhetbyt 's suggestion of https://ci.nodejs.org/job/node-test-linter/ : what would be a good description?

[gibfahn]

OK so for the ones you've marked as "possibly already documented", I cannot seem to find anything on these.

Okay, then please do document them as well.

https://ci.nodejs.org/job/node-test-linter/ : what would be a good description?

Maybe something like:

The node-test-linter only runs the linter targets, which is useful for changes that only affect comments or documentation.

[morrme]

OK ongoing notes:

node-stress_single-test

allows you to specify a set of tests to pass to tools/test.py, and that there's a repeat option to run a test or tests in a loop. A couple of examples might be helpful as well
it's designed to allow one to run a single test over and over on a specific platform to confirm that the test is reliable.

node-test-linter

The node-test-linter only runs the linter targets, which is useful for changes that only affect comments or documentation.

---

For node-test-pull-request
I have found #2263
Is this info still valid/current? anything to add?

There is also information there about node-test-commit but I'm still hoping to find a better description.

For citgm-smoker

Possibly use some of the NPM description: https://www.npmjs.com/package/citgm

citgm is a simple tool for pulling down an arbitrary module from npm and testing it using a specific version of the node runtime.

The Node.js project uses citgm to smoketest our releases and controversial changes.

[gibfahn]

@morrme Sorry for forgetting about this! What you've got looks good.

For node-test-pull-request, maybe something like this would be helpful:

node-test-pull-request is the standard CI run we do to check Pull Requests. It triggers node-test-commit, which runs the build-ci and test-ci targets on all supported platforms.

I would suggest you order it:

• node-test-pull-request
• node-test-linter
• citgm-smoker
• node-stress-test

as that's (I think) the order in which they get used.

For citgm-smoker, maybe something like:

The citgm-smoker job uses CitGM to allow you to run npm install && npm test on a large selection of common modules. This is useful to check whether a change will cause breakage in the ecosystem. To test node ABI changes you can run citgm-abi-smoker.

I think what you have is ready for a PR, if you submit one then you should get lots of feedback from other collaborators, and we can hopefully get it landed.

[morrme]

@gibfahn did not forget about this! i think i've been over-obsessing about wording and i'm sure there will be changes during the review process, so I'll send over what I have.