-## 4 Test Suite
+# Test Suite
You execute the Test Suite using CTest with the command `$ ctest`.
-The build creates 6 tests: bashTests, bugfixTests, lensTests, tiffTests, unitTests and versionTests. You can run all tests or a subset. To list all available tests, execute ctest with the `-N` or `--show-only` option, which disables execution:
+The build creates 6 tests: bashTests, bugfixTests, lensTests, tiffTests, unitTests and versionTests. You can run all tests or a subset. To list all available tests, execute ctest with the `-N` or `--show-only` option, which disables execution:
```bash
.../main/build $ ctest -N
@@ -880,21 +852,21 @@ $ ctest -R bugfix # run only bugfixTests and display summary
$ ctest -R bugfix --verbose # run only bugfixTests and display all output
```
+Except for the `unitTests`, CMake needs to find a python3 interpreter in the system to be able to run the rest of the test targets with CTest:
+
| Name | Language | Location | Command
_(in build directory)_ | CMake Option to Build |
|:-- |:-- |:-- |:-- |:-- |
-| bashTests | python | tests/bash\_tests | $ ctest -R bash | -DEXIV2\_BUILD\_SAMPLES=On |
-| bugfixTests | python | tests/bugfixes | $ ctest -R bugfix | -DEXIV2\_BUILD\_SAMPLES=On |
-| lensTest | C++ | tests/lens_tests | $ ctest -R lens | -DEXIV2\_BUILD\_SAMPLES=On |
-| tiffTests | python | tests/tiff_test | $ ctest -R tiff | -DEXIV2\_BUILD\_SAMPLES=On |
-| unitTests | C++ | unitTests/ | $ ctest -R unit | -DEXIV2\_BUILD\_UNIT\_TESTS=On |
+| bashTests | python | tests/bash_tests | $ ctest -R bash | -DEXIV2_BUILD_SAMPLES=ON |
+| bugfixTests | python | tests/bugfixes | $ ctest -R bugfix | |
+| lensTest | C++ | tests/lens_tests | $ ctest -R lens | |
+| tiffTests | python | tests/tiff_test | $ ctest -R tiff | |
+| unitTests | C++ | unitTests/ | $ ctest -R unit | -DEXIV2_BUILD_UNIT_TESTS=ON |
| versionTests | C++ | src/version.cpp | $ ctest -R version | Always in library |
The term _**bashTests**_ is historical. These tests were originally bash scripts and have been rewritten in python.
Visual Studio Users will appreciate the python implementation as it avoids the installation of mingw/cygwin and special PATH settings.
-#### Environment Variables used by the test suite:
-
-If you build the code in the directory \
build, tests will run using the default values of Environment Variables.
+If you build the code in the directory `/build`, tests will run using the default values of Environment Variables.
| Variable | Default | Platforms | Purpose |
|:-- |:-- |:-- |:-- |
@@ -909,9 +881,9 @@ If you build the code in the directory \build, tests will run using t
The Variable EXIV2\_PORT or EXIV2\_HTTP can be set to None to skip http tests. The http server is started with the command `python3 -m http.server $port`. On Windows, you will need to run this manually _**once**_ to authorise the firewall to permit python to use the port.
[TOC](#TOC)
-
+
-### 4.1 Running tests on Unix-like systems
+## Running tests on Unix-like systems
You can run tests directly from the build:
@@ -945,9 +917,9 @@ $
```
[TOC](#TOC)
-
+
-### 4.2 Running tests on Visual Studio builds from cmd.exe
+## Running tests on Visual Studio builds from cmd.exe
**Caution:** _The python3 interpreter must be on the PATH, build for DOS, and called python3.exe._ I copied the python.exe program:
@@ -969,9 +941,9 @@ Visual Studio can build different configs as follows:
> cmake --build . --config Release # or Debug or MinSizeRel or RelWithDebInfo
> ctest -C Release
```
-The default for **cmake** config option `--config` is `Release`. **ctest** does not have a default for config option `-C`.
+The default for **CMake** config option `--config` is `Release`. **ctest** does not have a default for config option `-C`.
-##### Running tests from cmd.exe
+### Running tests from cmd.exe
You can build with Visual Studio using Conan. The is described in detail in [README-CONAN.md](README-CONAN.md)
@@ -996,11 +968,11 @@ set EXIV2_PORT=
```
[TOC](#TOC)
-
+
-### 4.3 Unit Tests
+## Unit Tests
-The code for the unit tests is in `
/unitTests`. To include unit tests in the build, use the *cmake* option `-DEXIV2_BUILD_UNIT_TESTS=On`.
+The code for the unit tests is in `/unitTests`. To include unit tests in the build, use the *CMake* option `-DEXIV2_BUILD_UNIT_TESTS=On`.
There is a discussion on the web about installing GTest: [https://github.com/Exiv2/exiv2/issues/575](https://github.com/Exiv2/exiv2/issues/575)
@@ -1016,9 +988,9 @@ $ popd
```
[TOC](#TOC)
-
+
-### 4.4 Bugfix Tests
+## Bugfix Tests
You can run the bugfix tests from the build directory:
@@ -1057,9 +1029,9 @@ $ ctest -R bugfix --verbose 2>&1 | grep FAIL
```
[TOC](#TOC)
-
+
-### 4.5 Fuzzing
+## Fuzzing
The code for the fuzzers is in `exiv2dir/fuzz`
@@ -1086,24 +1058,24 @@ mkdir corpus
For more information about fuzzing see [`fuzz/README.md`](fuzz/README.md).
[TOC](#TOC)
-
+
-### 4.5.1 OSS-Fuzz
+### OSS-Fuzz
Exiv2 is enrolled in [OSS-Fuzz](https://google.github.io/oss-fuzz/), which is a fuzzing service for open-source projects, run by Google.
The build script used by OSS-Fuzz to build Exiv2 can be found [here](https://github.com/google/oss-fuzz/tree/master/projects/exiv2/build.sh). It uses the same fuzz target ([`fuzz-read-print-write`](fuzz/fuzz-read-print-write.cpp)) as mentioned above, but with a slightly different build configuration to integrate with OSS-Fuzz. In particular, it uses the CMake option `-DEXIV2_TEAM_OSS_FUZZ=ON`, which builds the fuzz target without adding the `-fsanitize=fuzzer` flag, so that OSS-Fuzz can control the sanitizer flags itself.
[TOC](#TOC)
-