Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-add benchmark tests #3449

Closed
ocelotl opened this issue Sep 26, 2023 · 2 comments · Fixed by #3450
Closed

Re-add benchmark tests #3449

ocelotl opened this issue Sep 26, 2023 · 2 comments · Fixed by #3450
Assignees
Labels

Comments

@ocelotl
Copy link
Contributor

ocelotl commented Sep 26, 2023

Benchmark tests were removed in #2999. There is an ongoing effort here to add new benchmarking infrastructure that would allow us to run these tests in separate hardware to make the results reliable.

Re-add benchmark tests along with a Github workflow that will run them in the new benchmarking infrastructure.

@ocelotl ocelotl added the tests label Sep 26, 2023
@ocelotl ocelotl self-assigned this Sep 26, 2023
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Sep 28, 2023
ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Oct 11, 2023
@aabmass
Copy link
Member

aabmass commented Nov 3, 2023

@ocelotl can we remove this stuff big comment block now? IIUC your PR replaces the benchmarking workflow we had to comment out

# - name: Find and merge benchmarks
# id: find_and_merge_benchmarks
# run: >-
# jq -s '.[0].benchmarks = ([.[].benchmarks] | add)
# | if .[0].benchmarks == null then null else .[0] end'
# $(find . -name '*${{ matrix.package }}*-benchmark.json') > output.json
# && echo "json_plaintext=$(cat output.json)" >> $GITHUB_OUTPUT
# - name: Report on benchmark results
# if: steps.find_and_merge_benchmarks.outputs.json_plaintext != 'null'
# uses: rhysd/github-action-benchmark@v1
# with:
# name: OpenTelemetry Python Benchmarks - Python ${{ env[matrix.python-version ]}} - ${{ matrix.package }}
# tool: pytest
# output-file-path: output.json
# github-token: ${{ secrets.GITHUB_TOKEN }}
# max-items-in-chart: 100
# # Alert with a commit comment on possible performance regression
# alert-threshold: 200%
# fail-on-alert: true
# # Make a commit on `gh-pages` with benchmarks from previous step
# auto-push: ${{ github.ref == 'refs/heads/main' }}
# gh-pages-branch: gh-pages
# benchmark-data-dir-path: benchmarks

@aabmass aabmass reopened this Nov 3, 2023
@ocelotl
Copy link
Contributor Author

ocelotl commented Nov 22, 2023

Fixed in #3511

@ocelotl ocelotl closed this as completed Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants