Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history 
Resolves #329.
  • Loading branch information
@aantron
aantron committed Oct 21, 2020
1 parent 556a5b5 commit 1f171b0
Showing 1 changed file with 16 additions and 84 deletions.
100 changes: 16 additions & 84 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ details of how it is generated are in the [worked example](#Example).
- [**Ocamlfind, Ocamlbuild, and OASIS**](#Ocamlbuild)
- [**Sending to Coveralls**](#Coveralls)
- [**Controlling coverage with `[@coverage off]`**](#Exclusion)
- [**Real-world example examined**](#Example)   ([repo][markupml], [report][gh-pages-report], [coveralls][markupml-coveralls])
- [**Other topics**](#Other)
- [**Bisect_ppx users**](#Users)
- [**Contributing**](#Contributing)
Expand All @@ -51,7 +50,7 @@ details of how it is generated are in the [worked example](#Example).
Refer to [**aantron/bisect-starter-dune**][dune-repo], which produces
[this report][dune-report].

1. [Depend on Bisect_ppx](https://github.com/aantron/bisect-starter-dune/blob/master/bisect-starter-dune.opam#L10)
1. [Depend on Bisect_ppx](https://github.com/aantron/bisect-starter-dune/blob/03cb827553d1264559eab19fdaa8c0056c9b2019/bisect-starter-dune.opam#L10-L11)
in your `opam` file:

```
Expand All @@ -62,7 +61,7 @@ Refer to [**aantron/bisect-starter-dune**][dune-repo], which produces
```

2. [Mark the code under test for preprocessing by
`bisect_ppx`](https://github.com/aantron/bisect-starter-dune/blob/master/dune) in your `dune` file:
`bisect_ppx`](https://github.com/aantron/bisect-starter-dune/blob/03cb827553d1264559eab19fdaa8c0056c9b2019/dune#L4) in your `dune` file:

```ocaml
(library
Expand All @@ -75,6 +74,7 @@ Refer to [**aantron/bisect-starter-dune**][dune-repo], which produces
`bisect0123456789.coverage`:

```
find . -name '*.coverage' | xargs rm -f
dune runtest --instrument-with bisect_ppx --force
```

Expand Down Expand Up @@ -114,21 +114,24 @@ report][esy-report].

The instructions are the same as for regular [Dune](#Dune) usage, but...

1. [Depend on Bisect_ppx in `package.json`](https://github.com/aantron/bisect-starter-esy/blob/master/package.json#L5),
1. [Depend on Bisect_ppx in `package.json`](https://github.com/aantron/bisect-starter-esy/blob/fc9707a641ec598b6849087841d63fa140bd7118/package.json#L8),
instead of in an `opam` file:

```json
"devDependencies": {
"@opam/bisect_ppx": "^2.0.0",
"@opam/bisect_ppx": "^2.5.0"
},
"dependencies": {
"@opam/dune": "^2.7.0"
}
```

2. Use the `esy` command for the build and for running binaries:

```
esy install
BISECT_ENABLE=yes esy dune runtest --force
esy dune exec bisect-ppx-report -- html
esy dune runtest --instrument-with bisect_ppx --force
esy bisect-ppx-report html
```

[esy-repo]: https://github.com/aantron/bisect-starter-esy
Expand Down Expand Up @@ -260,16 +263,17 @@ Refer to [**aantron/bisect-starter-jsoo**][jsoo-repo], which produces
[this report][jsoo-report].

1. Follow the [Dune instructions](#Dune) above, except that [the final test
script must be linked with `bisect_ppx.runtime`](https://github.com/aantron/bisect-starter-jsoo/blob/master/dune#L9)
script must be linked with `bisect_ppx.runtime`](https://github.com/aantron/bisect-starter-jsoo/blob/dcb2688017c9f322a992bbacc24f6d86ce4c2dc6/dune#L10)
(but not instrumented):

```scheme
(executable
(name my_tester)
(modes js)
(libraries bisect_ppx.runtime))
```

2. If the tests will run on Node, [call this function](https://github.com/aantron/bisect-starter-jsoo/blob/master/tester.ml#L3)
2. If the tests will run on Node, [call this function](https://github.com/aantron/bisect-starter-jsoo/blob/dcb2688017c9f322a992bbacc24f6d86ce4c2dc6/tester.ml#L3)
at the end of testing to write `bisect0123456789.coverage`:

```ocaml
Expand All @@ -283,14 +287,15 @@ at the end of testing to write `bisect0123456789.coverage`:
```

to get binary coverage data in a string option. Upload this string or
otherwise extract it from the browser to create an `.coverage` file.
otherwise extract it from the browser to create a `.coverage` file.

3. Build the usual Js_of_ocaml target, including the instrumented code under
test, then run the reporter to generate the [coverage report][jsoo-report] in
`_coverage/index.html`:

```
BISECT_ENABLE=yes dune build my_tester.bc.js
dune build my_tester.bc.js --instrument-with bisect_ppx
node _build/default/my_tester.bc.js # or in the browser
bisect-ppx-report html
```

Expand Down Expand Up @@ -411,79 +416,6 @@ exclude files by not preprocessing with Bisect_ppx to begin with.
<br>
<a id="Example"></a>
## Real-world example examined
Refer to:
- [**aantron/markup.ml**][markupml], which produces
[this local report][gh-pages-report], and
[this report on Coveralls][markupml-coveralls].
- The [Dune instructions](#Dune) and [Coveralls instructions](#Coveralls) above.
The details:
1. The project [depeds on package `bisect_ppx`](https://github.com/aantron/markup.ml/blob/33e40c49827fca4e10cc6a9c64a073f30d797f5b/markup.opam#L21),
so that Bisect_ppx is installed by `opam pin --dev-repo markup` and `opam install .`
2. There are three libraries in [`src/`](https://github.com/aantron/markup.ml/tree/33e40c49827fca4e10cc6a9c64a073f30d797f5b/src), each set to have its
sources preprocessed by Bisect_ppx:
- [`markup`](https://github.com/aantron/markup.ml/blob/33e40c49827fca4e10cc6a9c64a073f30d797f5b/src/dune#L5)
- [`markup-lwt`](https://github.com/aantron/markup.ml/blob/33e40c49827fca4e10cc6a9c64a073f30d797f5b/src/lwt/dune#L5)
- [`markup-lwt.unix`](https://github.com/aantron/markup.ml/blob/33e40c49827fca4e10cc6a9c64a073f30d797f5b/src/lwt_unix/dune#L5)
Because of the `--conditional` flag, preprocessing is enabled only when
`BISECT_ENABLE=yes` is set in the environment, so it is off by default.
3. A coverage build is triggered by running [`make coverage`](https://github.com/aantron/markup.ml/blob/33e40c49827fca4e10cc6a9c64a073f30d797f5b/Makefile#L19-L25). This target...
- Depends on `make clean`. This is a workaround until
[ocaml/dune#57][dune-57] is solved. The problem is that doing a coverage
build, after normal builds, should force all sources to be recompiled, so
that they can be instrumented by the Bisect_ppx preprocessor. However, Dune
doesn't know about this &mdash; it doesn't know that the behavior of the
preprocessor depends on the `BISECT_ENABLE` environment variable.
Indeed, the preprocessor shouldn't read this environment variable. The
preprocessor should just be turned off by Dune when not building for
coverage. However, Dune does not currently have the ability to
conditionally turn off a preprocessor.
In any case, to deal with this problem, the project always does a clean
build when building for coverage.
- Does a fresh build with `BISECT_ENABLE=yes`, causing the sources of the
three libraries mentioned above to be instrumented.
- Runs the test suite. `bisect*.coverage` files with coverage data are
produced as a side effect.
- Runs `bisect-ppx-report` to generate both the typical HTML report in
`_coverage/index.html`, and also a textual summary in the terminal for very
fast iteration.
4. `make coverage` is also [used in Travis](https://github.com/aantron/markup.ml/blob/33e40c49827fca4e10cc6a9c64a073f30d797f5b/.travis.yml#L33-L35)
to submit coverage reports to Coveralls. At the end of `make coverage`, the
`bisect*.coverage` files are still present, so `.travis.yml` runs
`bisect-ppx-report` again to generate the Coveralls report. This follows the
[Coveralls](#Coveralls) instructions exactly.
Coveralls can be configured to [leave comments](https://github.com/aantron/markup.ml/pull/47#issuecomment-521707675)
about changes in coverage. It is usually configured to at least add an
additional check to branches and PRs &mdash; see the "3 checks passed" in
the hidden Details of the [linked PR](https://github.com/aantron/markup.ml/pull/47).
5. During release, `(preprocess (pps bisect_ppx))` is [removed from all libraries that are being released](https://github.com/aantron/markup.ml/commit/ea68bebf5c3a19f56350393e359d444f864154e3#diff-d218652a79a651b9be8eee7641ea0893L5). This is typically in a one-commit release branch off master, which is what ends up being tagged.
This won't be necessary after [ocaml/dune#57][dune-57] is addressed.
[dune-57]: https://github.com/ocaml/dune/issues/57
<br>
<a id="Other"></a>
Expand Down

0 comments on commit 1f171b0

Please sign in to comment.