BuildKit Benchmarks

This repository contains a set of benchmarks for BuildKit that run on GitHub Actions public runners. Results are published on GitHub Pages.

---

• Usage
• License

Usage

To run locally, you can use the following command:

make test

This runs all tests and benchmarks from ./test package with BuildKit changes from default branch. You can also specify a commit to test or multiple references and tweak the benchmark settings:

# run only tests
TEST_TYPES=test make test

# run only benchmarks
TEST_TYPES=benchmark make test
# or
make bench

# run a specific benchmark
TEST_BENCH_REGEXP=/BenchmarkBuildLocal$ make bench

# run all benchmarks 3 times (default 1)
TEST_BENCH_RUN=3 make bench

# run 5 iterations of each benchmark (default 1x)
TEST_BENCH_TIME=5x make bench

# run all with master, v0.9.3 and v0.16.0 git references
BUILDKIT_REFS=master,v0.9.3,v0.16.0 make bench

Note

Set TEST_KEEP_CACHE=1 for the test framework to keep external dependant images in a docker volume if you are repeatedly calling make test or make bench. This helps to avoid rate limiting on the remote registry side.

After running the tests, you can generate the HTML report and serve the website with:

make gen

...
[+] Running 1/0
 ✔ Container website-serve-1  Created                                                                                                                                                                     0.0s 
Attaching to serve-1
serve-1  | Running generateResults.js...
serve-1  | /src/public/results.json has been generated successfully.
serve-1  |  INFO  Starting development server...
 DONE  Compiled successfully in 947ms2:39:29 PM
serve-1  |
                                                                                                                                                                                                               
serve-1  |   App running at:
serve-1  |   - Local:   http://localhost:8080/
serve-1  |
serve-1  |   It seems you are running Vue CLI inside a container.
serve-1  |   Access the dev server via http://localhost:<your container's external mapped port>/
serve-1  |
serve-1  |   Note that the development build is not optimized.
serve-1  |   To create a production build, run yarn build.
serve-1  |
Build finished at 14:39:29 by 0.000s

Then open http://localhost:8080 in your browser.

License

This project is licensed under the Apache License, Version 2.0 - see the LICENSE file for details.