[Perf] Linux/x64: 3 Regressions on 3/29/2023 3:17:08 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	80fe2a1a488e641ec99f6ffd7610561ea501ce83
Compare	3f5cb1805ea040e0869c1da8b8a7476a223ce4d2
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadReturnBytes - Duration of single invocation	106.86 μs	163.24 μs	1.53	0.09	False
ReadReturnBytes - Duration of single invocation	1.06 ms	1.57 ms	1.49	0.09	False
ReadSpanEmptyLoop - Duration of single invocation	18.97 μs	27.14 μs	1.43	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 163.24228129650297 > 112.67686668162212.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/29/2023 12:02:44 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 12:39:01 PM and 3/30/2023 4:24:24 AM.
IsRegressionStdDev: Marked as regression because -3.0183718615856705 (T) = (0 -162875.4830567881) / Math.Sqrt((6031247332.277068 / (54)) + (178983.3607033227 / (3))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (54) + (3) - 2, .025) and -0.243630071932174 = (130967.79077055892 - 162875.4830567881) / 130967.79077055892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.572778571875 > 1.1395093967395833.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/16/2023 1:49:47 PM, 3/17/2023 1:34:47 AM, 3/27/2023 2:09:01 AM, 3/30/2023 4:24:24 AM falls between 3/21/2023 12:39:01 PM and 3/30/2023 4:24:24 AM.
IsRegressionStdDev: Marked as regression because -2.259491713379494 (T) = (0 -2034558.9004839286) / Math.Sqrt((687085248307.2562 / (43)) + (1141267533315.1567 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.53084773828341 = (1329040.6678624658 - 2034558.9004839286) / 1329040.6678624658 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.141638875000005 > 19.80489812904716.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/25/2023 7:34:08 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 12:39:01 PM and 3/30/2023 4:24:24 AM.
IsRegressionStdDev: Marked as regression because -2.1247047816971563 (T) = (0 -35253.37815510913) / Math.Sqrt((137455893.1576764 / (38)) + (653359547.911208 / (19))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (19) - 2, .025) and -0.5912136720971564 = (22155.024666578298 - 35253.37815510913) / 22155.024666578298 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

dotnet/runtime#83512, same issue as #15196