v1.6.3

• Improve: Tweak number of retries for more robust automatic git push

v1.6.2

• Fix: Retry logic for git push did not work properly since stderr output was not included in error message

v1.6.1

• Fix: Time unit of mean time in pytest benchmark results were always sec. Now time units are converted to msec, usec and nsec if necessary
• Fix: Detecting rejection by remote on git push was not sufficient
• Improve: Add a small link at right bottom of dashboard page to show this action provided the page
• Improve: Showed at least 1 significant digit for threshold float values like 2.0
• Improve: Updated dependencies

v1.6.0

• New: fail-threshold input was added. Format is the same as alert-threshold, but you can give different thresholds to sending a commit comment and making the workflow fail by giving different value to fail-threshold from alert-threshold. This value is optional. If omitted, fail-threshold value is the same as alert-threshold
• Improve: Retry logic was improved on git push failed due to remote branch updates after git pull. Now this action retries entire process to update gh-pages branch when the remote rejected automatic git push. Previously this action tried to rebase the local onto the remote but it sometimes failed due to conflicts

v1.5.0

• New: Added max-items-in-chart input was added to limit the number of data points in a graph chart.
• New: Supported Google C++ Benchmark Framework for C++ projects. Please check the example project and the example workflow to know the setup
• Fix: Fix the order of graphs in the default index.html. To apply this fix, please remove index.html in your GitHub Pages branch and run your benchmark workflow again
• Improve: Use the actions marketplace URL for the link to this action in commit comment
• Improve: Updated dependencies
• Dev: Added Many tests for checking the updates on a new benchmark result
• Dev: Changed directory structure. Sources are now put in src/ directory

v1.4.0

• New: external-data-json-path input was added to support to put benchmark data externally rather than Git branch
  • By using this input and actions/cache, you no longer need to use Git branch for this action if you only want performance alerts. Benchmark data is stored as workflow cache.
  • By this input, minimal setup for this action is much easier. Please read 'How to use' section in README.md.

v1.3.2

• Improve: Styles in alert commit comment were improved
• Fix: When benchmark name (with name input) contained spaces, URL for the workflow which detected performance regression was broken

v1.3.1

• Fix: git push sometimes failed in the situation where prepush hook is set and runs unexpectedly. Now git push is run with --no-verify for pushing auto generated commit to remote.

v1.3.0

• New: Alert feature was added 🎉
  • With this feature enabled, you can get alert commit comment or make workflow fail when possible performance regression is detected like this
  • comment-on-alert input was added to enable commit comment on alert. github-token input is necessary as well to use GitHub API. Unlike deploying GitHub Pages, secrets.GITHUB_TOKEN is sufficient for this purpose (if you don't use GitHub Pages). The input is set to false by default.
  • fail-on-alert input was added to mark running workflow fail on alert. The input is set to false by default.
  • alert-threshold input was added to specify the threshold to check alerts. When current result gets worse than previous exceeding the threshold. Value is ratio such as "200%". For example, when benchmark gets result 230 ns/iter and previous one was 100ns/iter, it means 230% worse and an alert will happen.
  • Please read documentation for setup
• New: alert-comment-cc-users input was added to specify users mentioned in an alert commit comment so that they can easily notice it via GitHub notification
• New: skip-fetch-gh-pages input was added to skip git pull which is automatically executed on public repo or when you set github-token on private repo.
• Improve: E2E checks on CI were added
• Improve: Updated dependencies

v1.2.0

• New: Support pytest-benchmark for Python projects which use pytest
  • Benchmark value is how long one iteration takes (seconds/iter)
• Improve: Show more extra data in tooltip which are specific to tools
  • Go
    • Iterations
    • Number of CPUs used
  • Benchmark.js
    • Number of samples
  • pytest-benchmark
    • Mean time
    • Number of rounds

For reflecting the extra data improvement, please refresh your index.html. Remove current index.html in GitHub Pages branch and push the change to remote, then re-run your benchmark workflow.