Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 233 Improvements on 1/28/2023 4:01:39 AM #12313

Closed
performanceautofiler bot opened this issue Jan 31, 2023 · 9 comments
Closed

[Perf] Linux/x64: 233 Improvements on 1/28/2023 4:01:39 AM #12313

performanceautofiler bot opened this issue Jan 31, 2023 · 9 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanBenchmark - Duration of single invocation 39.61 ns 27.29 ns 0.69 0.04 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 34.00 ns 10.60 ns 0.31 0.02 False
SubtractBenchmark - Duration of single invocation 37.94 ns 24.35 ns 0.64 0.03 False
SubtractionOperatorBenchmark - Duration of single invocation 37.27 ns 25.20 ns 0.68 0.03 False
EqualsAnyBenchmark - Duration of single invocation 32.51 ns 10.53 ns 0.32 0.03 False
MinBenchmark - Duration of single invocation 41.22 ns 28.26 ns 0.69 0.05 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 33.97 ns 10.96 ns 0.32 0.01 False
GreaterThanBenchmark - Duration of single invocation 41.17 ns 27.41 ns 0.67 0.05 False
MultiplyOperatorBenchmark - Duration of single invocation 37.75 ns 25.23 ns 0.67 0.05 False
DivisionOperatorBenchmark - Duration of single invocation 37.31 ns 25.21 ns 0.68 0.04 False
DotBenchmark - Duration of single invocation 55.15 ns 20.01 ns 0.36 0.04 False
EqualityOperatorBenchmark - Duration of single invocation 6.68 ns 3.75 ns 0.56 0.12 False
NegateBenchmark - Duration of single invocation 36.51 ns 24.47 ns 0.67 0.06 False
MultiplyBenchmark - Duration of single invocation 37.23 ns 25.19 ns 0.68 0.06 False
InequalityOperatorBenchmark - Duration of single invocation 6.77 ns 3.88 ns 0.57 0.15 False
LessThanAnyBenchmark - Duration of single invocation 35.30 ns 10.87 ns 0.31 0.01 False
AddOperatorBenchmark - Duration of single invocation 37.06 ns 25.16 ns 0.68 0.06 False
AddBenchmark - Duration of single invocation 37.34 ns 25.19 ns 0.67 0.04 False
GreaterThanAnyBenchmark - Duration of single invocation 30.90 ns 10.12 ns 0.33 0.01 False
SumBenchmark - Duration of single invocation 25.72 ns 10.25 ns 0.40 0.07 False
DivideBenchmark - Duration of single invocation 36.86 ns 25.32 ns 0.69 0.01 False
LessThanOrEqualAllBenchmark - Duration of single invocation 6.91 ns 3.81 ns 0.55 0.13 False
GreaterThanOrEqualBenchmark - Duration of single invocation 40.42 ns 28.15 ns 0.70 0.06 False
LessThanOrEqualBenchmark - Duration of single invocation 40.58 ns 28.21 ns 0.70 0.04 False
MaxBenchmark - Duration of single invocation 41.96 ns 28.31 ns 0.67 0.05 False
AbsBenchmark - Duration of single invocation 41.42 ns 27.15 ns 0.66 0.07 False
LessThanAllBenchmark - Duration of single invocation 8.38 ns 3.78 ns 0.45 0.12 False
UnaryNegateOperatorBenchmark - Duration of single invocation 35.64 ns 24.42 ns 0.69 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.28809962831108 < 37.64716972796015.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 174.21500541281205 (T) = (0 -27.291010543885577) / Math.Sqrt((1.6147425395266093 / (299)) + (0.003659897638456832 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.325851568001148 = (40.48219835351045 - 27.291010543885577) / 40.48219835351045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.602907765804584 < 32.26122512744952.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 47.23295044774929 (T) = (0 -11.220971561498272) / Math.Sqrt((0.6445133398247838 / (299)) + (2.626047698459673 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6738376521630934 = (34.40302547463015 - 11.220971561498272) / 34.40302547463015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.34839962560075 < 35.430860652488775.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 139.40137759517813 (T) = (0 -25.09819394575296) / Math.Sqrt((0.24695749760813446 / (299)) + (0.07621568902365435 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3284574197159719 = (37.37394274408886 - 25.09819394575296) / 37.37394274408886 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.196436313729343 < 35.48315999466772.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 332.9810175776645 (T) = (0 -25.223698723635856) / Math.Sqrt((0.3595607909895176 / (299)) + (0.00135296133684018 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3246100047358 = (37.346864627108985 - 25.223698723635856) / 37.346864627108985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.526565957730156 < 30.812980849563388.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 365.8943577267745 (T) = (0 -10.516898639674247) / Math.Sqrt((1.0428714995600088 / (299)) + (0.000580114488827212 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6742854319712231 = (32.28869590734756 - 10.516898639674247) / 32.28869590734756 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.25851962189768 < 39.13499139387885.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 196.23837185603116 (T) = (0 -28.272887546677314) / Math.Sqrt((1.42588141088169 / (299)) + (0.00056907289528136 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3251936395036841 = (41.89777868406987 - 28.272887546677314) / 41.89777868406987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.959848009698247 < 32.276846590127974.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 411.51414251511267 (T) = (0 -10.834670201620055) / Math.Sqrt((0.6941060790611356 / (299)) + (0.0067303182521735085 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6728874626135788 = (33.12214899553347 - 10.834670201620055) / 33.12214899553347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.408404038161482 < 38.52301417447647.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 182.50394754638157 (T) = (0 -27.427581769159552) / Math.Sqrt((1.1310293367511524 / (299)) + (0.015737721868332798 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3245284100233798 = (40.60508565594726 - 27.427581769159552) / 40.60508565594726 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.225384780507603 < 35.75063052491087.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 158.47976445440676 (T) = (0 -25.140737647922755) / Math.Sqrt((0.27014673520723625 / (299)) + (0.05553255557378319 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3271984014688307 = (37.367238280659414 - 25.140737647922755) / 37.367238280659414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.212666095234844 < 35.53858864573029.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 278.31299040417036 (T) = (0 -25.208799185109573) / Math.Sqrt((0.3254542169681193 / (299)) + (0.008691413010740688 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3236473965078383 = (37.2716820412176 - 25.208799185109573) / 37.2716820412176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.007120654830537 < 52.39551447938253.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 171.36293120013957 (T) = (0 -20.017783654676954) / Math.Sqrt((10.69581249753709 / (299)) + (0.02337893472273667 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6249764214775364 = (53.37740025185617 - 20.017783654676954) / 53.37740025185617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.752633263136458 < 6.349437993890125.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 28.76321881209811 (T) = (0 -3.9595940109806764) / Math.Sqrt((0.19222109750135438 / (299)) + (0.11209525216873739 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4305502063784433 = (6.953368067443944 - 3.9595940109806764) / 6.953368067443944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.471611239774916 < 34.740109106206866.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 21.21988729896886 (T) = (0 -25.019814993924705) / Math.Sqrt((0.5282447782232812 / (299)) + (3.17954028793822 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3138326100943184 = (36.46313619970172 - 25.019814993924705) / 36.46313619970172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.193696590239277 < 35.312916642593684.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 183.76945558163484 (T) = (0 -25.21379814552072) / Math.Sqrt((1.2657898274830588 / (299)) + (0.0014245548623620567 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3249716546908268 = (37.352206497302596 - 25.21379814552072) / 37.352206497302596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.877504735199512 < 6.473592881870636.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 24.614295934013285 (T) = (0 -3.9554580544049496) / Math.Sqrt((0.1049873012507202 / (299)) + (0.18039331761760327 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4461022365754816 = (7.141133825762367 - 3.9554580544049496) / 7.141133825762367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.873654790776092 < 33.46304843225382.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 1118.0907354680785 (T) = (0 -10.870450283228609) / Math.Sqrt((0.12495414067355369 / (299)) + (0.0006200665768219261 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6913567010075968 = (35.22010786793777 - 10.870450283228609) / 35.22010786793777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.15759505740217 < 35.15558850577854.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 340.8735059192472 (T) = (0 -25.212128925491243) / Math.Sqrt((0.2857864978446533 / (299)) + (0.003266522951532702 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.323657207015555 = (37.27714583050357 - 25.212128925491243) / 37.27714583050357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.189699728666206 < 35.46776071025572.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 392.02422312183495 (T) = (0 -25.169565175968668) / Math.Sqrt((0.2635349807512298 / (299)) + (0.000939526891924086 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3262771500600955 = (37.35893057243609 - 25.169565175968668) / 37.35893057243609 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.119033425498857 < 29.306605399888166.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 52.933325399228245 (T) = (0 -10.54440115896496) / Math.Sqrt((0.7564896842560509 / (299)) + (1.7759537070924458 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6702760626489253 = (31.979483332863925 - 10.54440115896496) / 31.979483332863925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SumBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.249831038643164 < 24.492129481137074.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/6/2022 2:12:43 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 108.43988845009898 (T) = (0 -10.358606740832526) / Math.Sqrt((4.148296203482592 / (299)) + (0.09791862972115105 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6123842364989648 = (26.723904743375737 - 10.358606740832526) / 26.723904743375737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.324375501239484 < 35.1155016043631.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 416.63568265603294 (T) = (0 -25.27690841063279) / Math.Sqrt((0.17618758607303125 / (299)) + (0.002391462748304085 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3188673056225711 = (37.11010882209445 - 25.27690841063279) / 37.11010882209445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.810245407891382 < 6.590775037271093.
IsChangePoint: Marked as a change because one of 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 67.84047852870015 (T) = (0 -3.919669181974838) / Math.Sqrt((0.18576920477339387 / (299)) + (0.021446709305964883 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4674018312136954 = (7.359524331274099 - 3.919669181974838) / 7.359524331274099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.149144296776097 < 38.39130913786725.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 16.969985518555053 (T) = (0 -28.95784284297886) / Math.Sqrt((1.0286215937339362 / (299)) + (5.331647428649572 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2904960571484769 = (40.81420989233153 - 28.95784284297886) / 40.81420989233153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.207750296954146 < 38.53744718825063.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 159.86114451472997 (T) = (0 -28.399723316485485) / Math.Sqrt((0.9354309568702427 / (299)) + (0.034918933222637456 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3088630172765296 = (41.09130899720417 - 28.399723316485485) / 41.09130899720417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.312533371031016 < 39.88075263961826.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 139.36747151012975 (T) = (0 -28.459312017222206) / Math.Sqrt((1.1317536760136897 / (299)) + (0.07264941439212749 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.33295831403352366 = (42.66496774633736 - 28.459312017222206) / 42.66496774633736 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.14666380125209 < 39.45364924282829.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 218.9459736834114 (T) = (0 -26.595805425135914) / Math.Sqrt((0.5352062993819386 / (299)) + (0.034328117476468356 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3658423000591002 = (41.938788139944535 - 26.595805425135914) / 41.938788139944535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.779485429290304 < 7.957533268469161.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/6/2022 2:12:43 AM, 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 36.826078118961135 (T) = (0 -3.9191545124627796) / Math.Sqrt((0.3223924537576818 / (299)) + (0.12359614133148447 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5104562644981308 = (8.005729066157798 - 3.9191545124627796) / 8.005729066157798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.420825644684395 < 33.945159379908986.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 181.64875138380967 (T) = (0 -24.437170707959897) / Math.Sqrt((1.2658776390700244 / (299)) + (0.000635126170307228 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.327481826802634 = (36.336818366971094 - 24.437170707959897) / 36.336818366971094 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSpanEmptyLoop - Duration of single invocation 5.27 μs 4.39 μs 0.83 0.00 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 28.06 μs 24.25 μs 0.86 0.00 False
ReadSpanEmptyLoop - Duration of single invocation 33.84 μs 27.91 μs 0.82 0.00 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 3.75 μs 3.12 μs 0.83 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 6.26 μs 5.23 μs 0.84 0.00 False
ReadReturnBytes - Duration of single invocation 36.35 μs 30.42 μs 0.84 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 6.62 μs 5.56 μs 0.84 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 31.95 μs 26.70 μs 0.84 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 30.09 μs 25.96 μs 0.86 0.00 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 22.26 μs 19.15 μs 0.86 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 3.46 μs 2.96 μs 0.85 0.00 False
ReadReturnBytes - Duration of single invocation 7.47 μs 7.05 μs 0.94 0.03 False
ReadSpanEmptyLoop - Duration of single invocation 6.27 μs 5.22 μs 0.83 0.00 False
ReadSpanEmptyLoop - Duration of single invocation 22.32 μs 19.14 μs 0.86 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 15.11 μs 13.04 μs 0.86 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 35.16 μs 29.48 μs 0.84 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 35.97 μs 30.93 μs 0.86 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 31.07 μs 25.69 μs 0.83 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 23.37 μs 20.37 μs 0.87 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 341.44 μs 285.00 μs 0.83 0.00 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 6.34 μs 5.79 μs 0.91 0.02 False
ReadSpanEmptyLoop - Duration of single invocation 36.21 μs 30.80 μs 0.85 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 5.27 μs 4.53 μs 0.86 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 3.82 μs 3.24 μs 0.85 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 355.95 μs 297.79 μs 0.84 0.00 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 33.49 μs 27.93 μs 0.83 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 38.21 μs 33.24 μs 0.87 0.00 False
ReadSpanEmptyLoop - Duration of single invocation 3.75 μs 3.10 μs 0.83 0.00 False
ReadReturnBytes - Duration of single invocation 23.72 μs 20.52 μs 0.87 0.02 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 7.73 μs 7.16 μs 0.93 0.02 False
ReadReturnBytes - Duration of single invocation 7.02 μs 5.92 μs 0.84 0.00 False
ReadReturnBytes - Duration of single invocation 15.79 μs 13.55 μs 0.86 0.00 False
ReadReturnBytes - Duration of single invocation 5.78 μs 4.94 μs 0.86 0.00 False
ReadReturnBytes - Duration of single invocation 4.03 μs 3.67 μs 0.91 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 3.38 μs 2.80 μs 0.83 0.00 False
ReadSpanEmptyLoop - Duration of single invocation 6.35 μs 5.72 μs 0.90 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 14.50 μs 12.29 μs 0.85 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 369.51 μs 311.18 μs 0.84 0.00 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 14.44 μs 12.30 μs 0.85 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 28.29 μs 24.24 μs 0.86 0.00 False
ReadSpanEmptyLoop - Duration of single invocation 327.30 μs 270.59 μs 0.83 0.00 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 7.15 μs 6.51 μs 0.91 0.02 False
ReadReturnBytes - Duration of single invocation 384.26 μs 326.51 μs 0.85 0.01 False
ReadReturnBytes - Duration of single invocation 355.39 μs 298.42 μs 0.84 0.00 False
ReadReturnBytes - Duration of single invocation 38.94 μs 33.45 μs 0.86 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 31.40 μs 25.66 μs 0.82 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 355.58 μs 298.09 μs 0.84 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.394840929649418 < 4.954911950336501.
IsChangePoint: Marked as a change because one of 1/9/2023 6:56:09 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 110.06222428833064 (T) = (0 -4392.786045292282) / Math.Sqrt((4226.312616291789 / (299)) + (320.9206120995187 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14154848324328825 = (5117.104413640653 - 4392.786045292282) / 5117.104413640653 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.248563420542634 < 26.704570795635814.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 62.75396523689179 (T) = (0 -24297.544026368854) / Math.Sqrt((85875.87912987 / (299)) + (32009.985566942276 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12742818266457495 = (27845.895940767754 - 24297.544026368854) / 27845.895940767754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.905237771577383 < 32.19752672278604.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 160.03158549071608 (T) = (0 -28006.15853115788) / Math.Sqrt((144146.1614685558 / (299)) + (6258.930794750656 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15630022873071822 = (33194.46026283112 - 28006.15853115788) / 33194.46026283112 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1209932629600052 < 3.5691561857721745.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 168.89165933550458 (T) = (0 -3126.8970859881624) / Math.Sqrt((1692.6807395020685 / (299)) + (73.78483588110515 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1596347375079499 = (3720.878557873212 - 3126.8970859881624) / 3720.878557873212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.225218921077932 < 5.955515767211277.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 167.2761034060723 (T) = (0 -5241.757792424994) / Math.Sqrt((4626.712323304568 / (299)) + (195.6510311766781 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1554365191873585 = (6206.469864623271 - 5241.757792424994) / 6206.469864623271 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.41869007955809 < 34.48401828105127.
IsChangePoint: Marked as a change because one of 1/9/2023 6:56:09 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 61.9165363629088 (T) = (0 -30505.795055379494) / Math.Sqrt((184049.75395633947 / (299)) + (69581.85240516801 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1446403686092582 = (35664.29129438769 - 30505.795055379494) / 35664.29129438769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.557519178660271 < 6.2994686551911245.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 110.62309362505785 (T) = (0 -5584.708971084627) / Math.Sqrt((12985.809038346031 / (299)) + (238.12665295390963 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13777904895343251 = (6477.120469301845 - 5584.708971084627) / 6477.120469301845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.703683785709607 < 30.527208504744735.
IsChangePoint: Marked as a change because one of 1/9/2023 6:56:09 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 181.77477709718355 (T) = (0 -26698.37566695382) / Math.Sqrt((131962.45157965252 / (299)) + (2911.9164499077783 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15319803235147111 = (31528.476180909358 - 26698.37566695382) / 31528.476180909358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.956418948057756 < 28.47074729235117.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 168.27575978449428 (T) = (0 -26009.536698715194) / Math.Sqrt((75981.24018734849 / (299)) + (2095.3762780980132 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12004336794964274 = (29557.74836097462 - 26009.536698715194) / 29557.74836097462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.146542847968515 < 21.15314491750904.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 101.7059533196299 (T) = (0 -19169.522802741696) / Math.Sqrt((151573.21902853486 / (299)) + (1228.8369946362832 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11657987148545126 = (21699.21443263334 - 19169.522802741696) / 21699.21443263334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.956223556069404 < 3.2894275825208217.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 132.6821748696757 (T) = (0 -2964.2140603055486) / Math.Sqrt((1440.951346963257 / (299)) + (78.4226675948168 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13399306753742932 = (3422.8525767992783 - 2964.2140603055486) / 3422.8525767992783 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.0468319250413085 < 7.0782481933543675.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/7/2022 6:04:36 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 42.27338377681863 (T) = (0 -7054.9520143828095) / Math.Sqrt((22834.570560362437 / (299)) + (77.83538293436075 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05189567604817479 = (7441.113637133126 - 7054.9520143828095) / 7441.113637133126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.220958151626973 < 5.970176745724112.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 130.72621556692192 (T) = (0 -5222.162668654526) / Math.Sqrt((5974.25533731384 / (299)) + (394.3174913446348 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15756937730580522 = (6198.923125507261 - 5222.162668654526) / 6198.923125507261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.13971013361893 < 21.162055357312436.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 51.77185552574255 (T) = (0 -19202.420422507468) / Math.Sqrt((152469.16879086534 / (299)) + (20501.534866860795 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11610523524562377 = (21724.781261538064 - 19202.420422507468) / 21724.781261538064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.041347636452244 < 14.39346948091755.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 160.66113475864174 (T) = (0 -13062.87371558605) / Math.Sqrt((13852.38378188888 / (299)) + (1113.9676418133613 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12999725441374432 = (15014.749989994076 - 13062.87371558605) / 15014.749989994076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.476546108490563 < 33.33158650904895.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 124.91380171852218 (T) = (0 -29549.26086712962) / Math.Sqrt((200867.7091200582 / (299)) + (9594.013524720514 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14244489056371473 = (34457.56493311999 - 29549.26086712962) / 34457.56493311999 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.928064808832474 < 34.171459273214225.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 99.25497867879946 (T) = (0 -30959.35703029771) / Math.Sqrt((346705.3625983984 / (299)) + (8364.091268764725 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1231731280924312 = (35308.403542588174 - 30959.35703029771) / 35308.403542588174 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.6925165596411 < 29.61854972605171.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 179.5072031562649 (T) = (0 -25699.027852578914) / Math.Sqrt((185432.99574708543 / (299)) + (1591.759414154498 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.16190361820078197 = (30663.57093370152 - 25699.027852578914) / 30663.57093370152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.374872236727587 < 22.212704036881266.
IsChangePoint: Marked as a change because one of 1/9/2023 12:42:28 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 80.41267279932205 (T) = (0 -20414.268946190834) / Math.Sqrt((149626.54303477128 / (299)) + (4295.548943828884 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10520479565415368 = (22814.459495360166 - 20414.268946190834) / 22814.459495360166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 285.00262219696964 < 317.9455966921543.
IsChangePoint: Marked as a change because one of 1/3/2023 9:28:36 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 56.27512362051715 (T) = (0 -283669.23782519903) / Math.Sqrt((24979333.55686832 / (299)) + (7351598.020973748 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14677127485131727 = (332465.64428051474 - 283669.23782519903) / 332465.64428051474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.785704569814217 < 6.01653198055194.
IsChangePoint: Marked as a change because one of 12/20/2022 1:34:12 PM, 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 22.447454714949032 (T) = (0 -5799.432472324478) / Math.Sqrt((13987.270979529356 / (299)) + (5294.324917123433 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0816818624511745 = (6315.275975931741 - 5799.432472324478) / 6315.275975931741 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.80245834155161 < 34.33820203196338.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 47.863374852705256 (T) = (0 -30939.067708937706) / Math.Sqrt((265311.2814059945 / (299)) + (85517.25331021057 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1258572224994788 = (35393.60903650463 - 30939.067708937706) / 35393.60903650463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.529891260353462 < 5.005124670296986.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 104.68815784366409 (T) = (0 -4540.095957692729) / Math.Sqrt((5103.961531436892 / (299)) + (206.34801971017387 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12128277076544812 = (5166.73146564748 - 4540.095957692729) / 5166.73146564748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2374848456598304 < 3.6195476784600262.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 32.35422973993618 (T) = (0 -3269.2427465341675) / Math.Sqrt((1406.9881901598753 / (299)) + (2530.955211808484 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13167630818380538 = (3765.0046605272123 - 3269.2427465341675) / 3765.0046605272123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 297.79112219551286 < 339.2409550366557.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 50.662424405662 (T) = (0 -299205.7188401125) / Math.Sqrt((14667435.326092195 / (299)) + (11182254.913980821 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1487844493302348 = (351504.05629301217 - 299205.7188401125) / 351504.05629301217 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.928328656082293 < 31.881835524910635.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 92.75874303112786 (T) = (0 -28050.951353475106) / Math.Sqrt((114249.0597054285 / (299)) + (28929.66709758673 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15360733120862016 = (33141.770230041 - 28050.951353475106) / 33141.770230041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.235239157801416 < 36.33549505646757.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 30.05302354093368 (T) = (0 -33360.16237971956) / Math.Sqrt((327183.727616528 / (299)) + (179254.56317053424 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1061841015976601 = (37323.30386978964 - 33360.16237971956) / 37323.30386978964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.10460639154546 < 3.5763874094946497.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 202.72511616603106 (T) = (0 -3107.99400615621) / Math.Sqrt((1785.3548790154398 / (299)) + (27.479236604748685 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15953876951746707 = (3697.9623728411857 - 3107.99400615621) / 3697.9623728411857 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.522888416291714 < 22.468244316444753.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 85.21745181517117 (T) = (0 -20570.655176314405) / Math.Sqrt((182032.60016534384 / (299)) + (3085.037079778328 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10995329077707656 = (23111.882739585777 - 20570.655176314405) / 23111.882739585777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.159919790713631 < 7.322354280048473.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 14.584681357865689 (T) = (0 -7206.771864222255) / Math.Sqrt((6924.3861574778875 / (299)) + (9034.243863735262 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.055542621481545436 = (7630.595120689648 - 7206.771864222255) / 7630.595120689648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.922160466439135 < 6.676132347024254.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 124.42938149729235 (T) = (0 -5953.500289659664) / Math.Sqrt((5903.305153048474 / (299)) + (473.5338901226072 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1420846794609034 = (6939.496413140874 - 5953.500289659664) / 6939.496413140874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.545552247243217 < 15.032716646283191.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 32.120310004718355 (T) = (0 -13693.641593466722) / Math.Sqrt((25473.14914076162 / (299)) + (39721.60010032066 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12480857814981004 = (15646.453166231693 - 13693.641593466722) / 15646.453166231693 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.9444181046731925 < 5.48507911868247.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 83.01331093092492 (T) = (0 -4956.069970396767) / Math.Sqrt((4832.208952277666 / (299)) + (670.0842082123006 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12820110885432653 = (5684.877579832378 - 4956.069970396767) / 5684.877579832378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6684401817777563 < 3.8331434408441463.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 22.62138648501337 (T) = (0 -3424.099822207533) / Math.Sqrt((2083.452798602131 / (299)) + (6706.189997636725 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14093203724201783 = (3985.831122388364 - 3424.099822207533) / 3985.831122388364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.803217726984554 < 3.205864003517085.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 144.69254295688708 (T) = (0 -2809.896130662571) / Math.Sqrt((1806.771676197908 / (299)) + (61.00530076143902 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1491506054277124 = (3302.4600459110325 - 2809.896130662571) / 3302.4600459110325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.721721811738243 < 6.0304441547265.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 66.8303776690353 (T) = (0 -5741.38701308728) / Math.Sqrt((7111.747690675683 / (299)) + (505.01838764980647 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08856949722803407 = (6299.314095398164 - 5741.38701308728) / 6299.314095398164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.288772296860964 < 13.760925847489563.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 219.871941067315 (T) = (0 -12283.171756094576) / Math.Sqrt((20231.83840139278 / (299)) + (180.41134570294537 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1409842824699655 = (14299.12341000339 - 12283.171756094576) / 14299.12341000339 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.17749693080356 < 349.8447440509466.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 43.3743784202059 (T) = (0 -313912.37025600526) / Math.Sqrt((18419394.568225976 / (299)) + (13162376.672908377 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13418978310036003 = (362564.8717568694 - 313912.37025600526) / 362564.8717568694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.297636942227719 < 13.737264129727665.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 23.21069067931832 (T) = (0 -12389.434375541845) / Math.Sqrt((20918.150821651892 / (299)) + (74016.0144470148 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13380299863690864 = (14303.252442625875 - 12389.434375541845) / 14303.252442625875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.235999358302053 < 26.881455715274384.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 175.46440152103438 (T) = (0 -24171.22658955575) / Math.Sqrt((101643.17258356982 / (299)) + (1223.1826630346236 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13358907686792182 = (27898.109250719845 - 24171.22658955575) / 27898.109250719845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 270.5864811422413 < 314.7845551868476.
IsChangePoint: Marked as a change because one of 1/9/2023 3:58:12 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 62.29716434391719 (T) = (0 -274375.07411634194) / Math.Sqrt((25278084.705585573 / (299)) + (6434837.331677096 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15667592517420525 = (325349.509526358 - 274375.07411634194) / 325349.509526358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.513290790550595 < 6.796608718031071.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 89.47343402681139 (T) = (0 -6524.306709735456) / Math.Sqrt((8848.136757174401 / (299)) + (109.56686469179542 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07940003701189205 = (7087.01604609964 - 6524.306709735456) / 7087.01604609964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 326.51035506810894 < 366.5567264117624.
IsChangePoint: Marked as a change because one of 1/9/2023 3:58:12 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 163.81652298617314 (T) = (0 -326599.61422043253) / Math.Sqrt((19072187.352415904 / (299)) + (477464.0617944153 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14105525433086707 = (380233.5550303713 - 326599.61422043253) / 380233.5550303713 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 298.4216347484277 < 337.8592210815747.
IsChangePoint: Marked as a change because one of 1/9/2023 12:42:28 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 56.526066933257574 (T) = (0 -301452.4713308017) / Math.Sqrt((31673281.384236462 / (299)) + (8130844.117613368 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1470332768130029 = (353416.4500632153 - 301452.4713308017) / 353416.4500632153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.446985641633525 < 37.0467595654836.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 124.76838464974864 (T) = (0 -33523.69976476027) / Math.Sqrt((308958.725368692 / (299)) + (4553.4344160652245 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12402691844394717 = (38270.23965760427 - 33523.69976476027) / 38270.23965760427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.655340871710532 < 29.728403781874995.
IsChangePoint: Marked as a change because one of 1/9/2023 6:56:09 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 169.06573711833178 (T) = (0 -25658.406268471448) / Math.Sqrt((186907.38441639993 / (299)) + (2899.7783859371953 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1641806368083638 = (30698.506637239156 - 25658.406268471448) / 30698.506637239156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 298.09046808176106 < 338.3348726609068.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 233.45942758101225 (T) = (0 -298527.1270128849) / Math.Sqrt((12447354.60325175 / (299)) + (97407.31929250936 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1494539251050779 = (350982.8988979409 - 298527.1270128849) / 350982.8988979409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiCharArray - Duration of single invocation 183.78 ns 151.91 ns 0.83 0.06 False
WriteAsciiCharArray - Duration of single invocation 10.12 ms 6.18 ms 0.61 0.02 False
WriteAsciiString - Duration of single invocation 21.83 ms 9.07 ms 0.42 0.02 False
WriteAsciiString - Duration of single invocation 40.09 μs 24.53 μs 0.61 0.03 False
WriteAsciiString - Duration of single invocation 168.40 ns 134.05 ns 0.80 0.01 False
WriteAsciiCharArray - Duration of single invocation 39.95 μs 24.53 μs 0.61 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 32)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 151.91391614578947 < 176.1569638263769.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 28.92617588627644 (T) = (0 -153.6893664840051) / Math.Sqrt((72.59173176284132 / (299)) + (17.90735332735802 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.20472439519984423 = (193.25296231439864 - 153.6893664840051) / 193.25296231439864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.176818883814102 < 9.624418124601936.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 306.9057496878377 (T) = (0 -6183918.003860723) / Math.Sqrt((47444991424.17624 / (299)) + (106415511.71273443 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3917087929940205 = (10166048.650116252 - 6183918.003860723) / 10166048.650116252 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.068960765625 < 20.758469863636368.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 335.6999002425481 (T) = (0 -9086211.143245297) / Math.Sqrt((455529203437.8704 / (299)) + (715110616.9015583 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5955562220604633 = (22465943.695649244 - 9086211.143245297) / 22465943.695649244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.533704656313077 < 38.17366791986262.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 262.57776499688464 (T) = (0 -24566.14993649174) / Math.Sqrt((708091.3336446127 / (299)) + (14770.26206776044 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.39435178730088377 = (40561.747597686895 - 24566.14993649174) / 40561.747597686895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 134.050046587544 < 159.69619404551412.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 92.80225379574229 (T) = (0 -133.9881704997415) / Math.Sqrt((52.20322711195013 / (299)) + (0.0910688118598444 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.22850611404426638 = (173.67366474169233 - 133.9881704997415) / 173.67366474169233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.526636390010847 < 37.948601214410985.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 338.7707501349872 (T) = (0 -24518.05130749401) / Math.Sqrt((666592.738055575 / (299)) + (95.94177391996193 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3952866144478581 = (40544.912504470965 - 24518.05130749401) / 40544.912504470965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Segment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadMultiSegmentSequence - Duration of single invocation 33.90 μs 27.92 μs 0.82 0.00 False
ReadSingleSegmentSequenceByN - Duration of single invocation 33.76 μs 28.02 μs 0.83 0.00 False
ReadMultiSegmentSequence - Duration of single invocation 3.77 ms 3.19 ms 0.85 0.00 False
ReadMultiSegmentSequence - Duration of single invocation 368.89 μs 313.19 μs 0.85 0.00 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 37.02 μs 30.43 μs 0.82 0.00 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 356.60 μs 300.15 μs 0.84 0.01 False
ReadMultiSegmentSequence - Duration of single invocation 374.66 μs 318.29 μs 0.85 0.00 False
ReadSingleSegmentSequence - Duration of single invocation 355.57 μs 298.74 μs 0.84 0.00 False
ReadSingleSegmentSequenceByN - Duration of single invocation 33.79 μs 27.93 μs 0.83 0.00 False
ReadMultiSegmentSequence - Duration of single invocation 35.33 μs 29.64 μs 0.84 0.01 False
ReadMultiSegmentSequence - Duration of single invocation 3.77 ms 3.19 ms 0.85 0.00 False
ReadSingleSegmentSequence - Duration of single invocation 33.56 μs 28.04 μs 0.84 0.00 False
ReadSingleSegmentSequenceByN - Duration of single invocation 3.65 ms 3.06 ms 0.84 0.00 False
ReadSingleSegmentSequenceByN - Duration of single invocation 355.98 μs 298.69 μs 0.84 0.00 False
ReadSingleSegmentSequenceByN - Duration of single invocation 358.65 μs 299.66 μs 0.84 0.00 False
ReadSingleSegmentSequenceByN - Duration of single invocation 3.68 ms 3.05 ms 0.83 0.00 False
ReadSingleSegmentSequence - Duration of single invocation 3.62 ms 3.06 ms 0.84 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Segment*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json4KB)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.91985639343734 < 32.16001294002242.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 176.57869760617328 (T) = (0 -28038.496773279832) / Math.Sqrt((145450.27019868305 / (299)) + (4162.550082970965 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15626597054238586 = (33231.43999691952 - 28038.496773279832) / 33231.43999691952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.01645098869154 < 32.09905532695862.
IsChangePoint: Marked as a change because one of 1/9/2023 6:56:09 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 189.81802767689967 (T) = (0 -27997.9643606344) / Math.Sqrt((130716.85016964878 / (299)) + (3371.6111642887035 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15603786320989763 = (33174.43181411068 - 27997.9643606344) / 33174.43181411068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1909803858333334 < 3.5878508530580357.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 148.40129091335226 (T) = (0 -3197814.545000416) / Math.Sqrt((1940235159.338567 / (299)) + (62029773.69307302 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13912958463212374 = (3714629.3889468745 - 3197814.545000416) / 3714629.3889468745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 313.188593125 < 350.2917875145349.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 145.62273885315523 (T) = (0 -313529.20398295164) / Math.Sqrt((18902595.98066746 / (299)) + (574642.217518186 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1363090151105009 = (363010.8562764085 - 313529.20398295164) / 363010.8562764085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.432599689464226 < 35.03381284709036.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 43.86786171102823 (T) = (0 -30493.822195723755) / Math.Sqrt((160312.93361018342 / (299)) + (194457.35691217176 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.16258461840528515 = (36414.21314432208 - 30493.822195723755) / 36414.21314432208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 300.15208301886787 < 339.0482844902408.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 130.7146599098423 (T) = (0 -300009.25119839865) / Math.Sqrt((12142793.581001882 / (299)) + (1323275.3937885822 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14877336170032399 = (352443.4477258215 - 300009.25119839865) / 352443.4477258215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 318.29194560860054 < 355.9363840823067.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 36.73198483947346 (T) = (0 -319266.7051527414) / Math.Sqrt((20547112.874307986 / (299)) + (19351916.393621866 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13461310523491127 = (368929.4431011774 - 319266.7051527414) / 368929.4431011774 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 298.7436684636118 < 337.95997617151073.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 208.36590652443283 (T) = (0 -298628.9676736929) / Math.Sqrt((13487142.891861944 / (299)) + (191736.1498999361 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1485655598451224 = (350736.3028671611 - 298628.9676736929) / 350736.3028671611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.927829531250005 < 32.14104882699591.
IsChangePoint: Marked as a change because one of 1/9/2023 6:56:09 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 154.68791176968566 (T) = (0 -28002.61479571775) / Math.Sqrt((142898.9913194765 / (299)) + (7276.4290723835975 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15716066485840768 = (33224.14323604565 - 28002.61479571775) / 33224.14323604565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.642043127756764 < 33.46418860702219.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 135.21320782605605 (T) = (0 -29593.27690475808) / Math.Sqrt((199298.9585687297 / (299)) + (7353.592133418398 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14306244383637154 = (34533.76117303391 - 29593.27690475808) / 34533.76117303391 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1908052575 < 3.5840501951634613.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 47.10435468112124 (T) = (0 -3206101.3048139364) / Math.Sqrt((1980373709.700902 / (299)) + (1174602101.3052888 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1352914386271039 = (3707724.7156239734 - 3206101.3048139364) / 3707724.7156239734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.041985120278465 < 31.974223516465056.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 182.53508529035216 (T) = (0 -27972.929309335075) / Math.Sqrt((127364.0886054781 / (299)) + (4048.432717112414 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15531552265635362 = (33116.42401350148 - 27972.929309335075) / 33116.42401350148 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.061333757440477 < 3.4873327112611605.
IsChangePoint: Marked as a change because one of 1/9/2023 6:56:09 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 165.74266527178895 (T) = (0 -3070137.8902926934) / Math.Sqrt((1454513235.799524 / (299)) + (64476024.948580645 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15024207127547598 = (3612955.8625018443 - 3070137.8902926934) / 3612955.8625018443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 298.68779842767293 < 338.55430505854065.
IsChangePoint: Marked as a change because one of 1/9/2023 3:58:12 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 63.03761800905028 (T) = (0 -299449.42483842815) / Math.Sqrt((12831384.72487838 / (299)) + (7031199.250908179 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14811116626376028 = (351512.32529377553 - 299449.42483842815) / 351512.32529377553 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 299.6559182109165 < 339.70479229439826.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 69.25706092420612 (T) = (0 -299881.6967444045) / Math.Sqrt((13698836.505562197 / (299)) + (5852563.48370536 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14934228432669824 = (352529.2150051751 - 299881.6967444045) / 352529.2150051751 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.046116028846154 < 3.476616461054258.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 129.66422143451496 (T) = (0 -3059435.2923687426) / Math.Sqrt((1489456404.8321517 / (299)) + (141364959.14497712 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15180456205166068 = (3606993.3360748426 - 3059435.2923687426) / 3606993.3360748426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.057139588541666 < 3.455461497422619.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 138.72219827800868 (T) = (0 -3047812.0685994555) / Math.Sqrt((1567817645.948462 / (299)) + (107344778.72382098 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.149871530065035 = (3585119.3982864893 - 3047812.0685994555) / 3585119.3982864893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 41.68 ms 34.24 ms 0.82 0.01 False
WriteStringsUtf16 - Duration of single invocation 57.78 ms 50.24 ms 0.87 0.00 False
WriteStringsUtf16 - Duration of single invocation 58.07 ms 50.23 ms 0.87 0.02 False
WriteStringsUtf16 - Duration of single invocation 56.63 ms 49.11 ms 0.87 0.00 False
WriteStringsUtf16 - Duration of single invocation 252.11 ms 194.64 ms 0.77 0.00 False
WriteStringsUtf16 - Duration of single invocation 40.24 ms 32.99 ms 0.82 0.01 False
WriteStringsUtf16 - Duration of single invocation 56.60 ms 49.00 ms 0.87 0.01 False
WriteStringsUtf16 - Duration of single invocation 41.57 ms 33.98 ms 0.82 0.01 False
WriteStringsUtf16 - Duration of single invocation 253.24 ms 197.03 ms 0.78 0.00 False
WriteStringsUtf16 - Duration of single invocation 40.11 ms 32.94 ms 0.82 0.01 False
WriteStringsUtf16 - Duration of single invocation 253.06 ms 195.81 ms 0.77 0.00 False
WriteStringsUtf16 - Duration of single invocation 253.03 ms 194.65 ms 0.77 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: NoneEscaped)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.244063296703295 < 39.591066703183756.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 6:48:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 143.56032279236547 (T) = (0 -34397923.19812568) / Math.Sqrt((631842735117.774 / (299)) + (9146580820.326893 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18465590539002655 = (42188228.78036568 - 34397923.19812568) / 42188228.78036568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.2363074375 < 55.03021005386904.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 151.74279617480303 (T) = (0 -50107382.11645854) / Math.Sqrt((677492128449.3256 / (299)) + (9769561366.456768 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14535303359310567 = (58629333.61493101 - 50107382.11645854) / 58629333.61493101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.23005759615385 < 55.17597716041666.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 130.2233030314347 (T) = (0 -50353189.414127536) / Math.Sqrt((763947657483.6206 / (299)) + (18721096594.67182 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14437536402220327 = (58849625.52134157 - 50353189.414127536) / 58849625.52134157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.108770788461534 < 53.96150575791666.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 42.65799763285815 (T) = (0 -49412948.50198135) / Math.Sqrt((635608569560.2865 / (299)) + (375080275098.24164 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14112048816787856 = (57531874.75222918 - 49412948.50198135) / 57531874.75222918 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 194.6416442142857 < 239.50308445702382.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 166.02640997532816 (T) = (0 -193666830.6953047) / Math.Sqrt((14785784941938.445 / (299)) + (887325495094.1635 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.23619513105710738 = (253555375.94744578 - 193666830.6953047) / 253555375.94744578 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.988461259615384 < 38.363127305595235.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 113.27325178666997 (T) = (0 -33347523.38452619) / Math.Sqrt((699497056224.8866 / (299)) + (24858243727.120594 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18723058203224888 = (41029500.67672126 - 33347523.38452619) / 41029500.67672126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.9998341875 < 53.749768619047614.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 134.218195811768 (T) = (0 -48926776.204041794) / Math.Sqrt((790789456949.1678 / (299)) + (14454307665.909094 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14719608019726665 = (57371659.61357132 - 48926776.204041794) / 57371659.61357132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.98181058163265 < 39.35563341414834.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 2:19:41 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 95.21445355860875 (T) = (0 -34250871.80494743) / Math.Sqrt((609798610535.7258 / (299)) + (49473339630.1884 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1835144302605183 = (41949145.30562487 - 34250871.80494743) / 41949145.30562487 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.02984657142858 < 241.00647643923077.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 129.13748587333035 (T) = (0 -195527797.09560442) / Math.Sqrt((15381922133138.387 / (299)) + (1777626595177.188 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.23362645070638502 = (255133801.6243216 - 195527797.09560442) / 255133801.6243216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.94287892857143 < 38.14341687910257.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 52.348071787108275 (T) = (0 -33304747.505813237) / Math.Sqrt((743856504219.8732 / (299)) + (198872338420.78644 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1839493914833217 = (40812110.374319464 - 33304747.505813237) / 40812110.374319464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 195.80630092307692 < 240.55734149791667.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 195.3673850309812 (T) = (0 -194655130.3153846) / Math.Sqrt((15151536598347.953 / (299)) + (491424167650.3037 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.23659234936470078 = (254981896.17748106 - 194655130.3153846) / 254981896.17748106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 194.6474025 < 239.73218229285717.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 114.4311874322514 (T) = (0 -193973699.76152182) / Math.Sqrt((15324558827060.205 / (299)) + (2453738139917.909 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2360462921659609 = (253907661.9072586 - 193973699.76152182) / 253907661.9072586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetChars - Duration of single invocation 158.89 ns 106.65 ns 0.67 0.05 False
GetString - Duration of single invocation 163.36 ns 106.14 ns 0.65 0.05 False
GetString - Duration of single invocation 6.64 μs 3.00 μs 0.45 0.06 False
GetChars - Duration of single invocation 4.05 μs 2.22 μs 0.55 0.05 False
GetChars - Duration of single invocation 196.91 ns 140.54 ns 0.71 0.01 False
GetChars - Duration of single invocation 6.63 μs 3.00 μs 0.45 0.05 False
GetString - Duration of single invocation 4.05 μs 2.25 μs 0.56 0.05 False
GetBytes - Duration of single invocation 5.79 μs 2.43 μs 0.42 0.02 False
GetBytes - Duration of single invocation 2.66 μs 1.63 μs 0.61 0.02 False
GetByteCount - Duration of single invocation 3.24 μs 791.48 ns 0.24 0.07 False
GetString - Duration of single invocation 172.49 ns 120.39 ns 0.70 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "ascii")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.64976399328812 < 153.24065218415407.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 56.43679646088301 (T) = (0 -105.37867038777038) / Math.Sqrt((57.637015276837246 / (299)) + (11.843939358198549 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3763353507694624 = (168.96688070709803 - 105.37867038777038) / 168.96688070709803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.14156748300714 < 155.1709134273132.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 81.84291490358338 (T) = (0 -105.54257105029318) / Math.Sqrt((57.044251306939756 / (299)) + (4.710930613457855 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3789288072568053 = (169.93634913917788 - 105.54257105029318) / 169.93634913917788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9967666498365064 < 6.30721555499746.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 194.7765808437925 (T) = (0 -2954.4743948636224) / Math.Sqrt((107731.96891991716 / (299)) + (757.5326038274599 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5773022448030432 = (6989.5672700863515 - 2954.4743948636224) / 6989.5672700863515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2237102863941556 < 3.845025301898691.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 99.41425598564766 (T) = (0 -2231.7087570136864) / Math.Sqrt((8420.640436728116 / (299)) + (3852.984906838584 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.46425945473854285 = (4165.652155232244 - 2231.7087570136864) / 4165.652155232244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 140.5388287481089 < 186.5765932177354.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 75.9112753138749 (T) = (0 -140.46618877789925) / Math.Sqrt((141.33958937825173 / (299)) + (3.989083937563236 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.33062650589896436 = (209.84725271583162 - 140.46618877789925) / 209.84725271583162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.999438517011834 < 6.296959585682965.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 185.94007758283752 (T) = (0 -2927.2100833861555) / Math.Sqrt((108455.46834401181 / (299)) + (1243.7733985648836 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5808139692173665 = (6983.0811821686 - 2927.2100833861555) / 6983.0811821686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2507332592263234 < 3.8524589432296534.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 67.63111842910756 (T) = (0 -2261.317350549879) / Math.Sqrt((9092.316590533102 / (299)) + (8380.650070080508 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4570627248313193 = (4164.969792960575 - 2261.317350549879) / 4164.969792960575 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.431610589791472 < 5.494839324664158.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 382.798903747825 (T) = (0 -2432.8000564106596) / Math.Sqrt((24498.75017814545 / (299)) + (8.544351551486933 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5886521613219869 = (5914.21621231602 - 2432.8000564106596) / 5914.21621231602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6312470726117305 < 2.523336438570872.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 330.5019410915215 (T) = (0 -1630.4616079626562) / Math.Sqrt((2682.943028297743 / (299)) + (8.180646179516152 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.387204005402815 = (2660.692338621474 - 1630.4616079626562) / 2660.692338621474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 791.4754347680392 < 3.0837538838669905.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 325.22378019016674 (T) = (0 -791.3554181095627) / Math.Sqrt((16589.171260552077 / (299)) + (1.318081844520171 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.7539657010930542 = (3216.443486234683 - 791.3554181095627) / 3216.443486234683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 120.3880495667706 < 163.82900391812638.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 71.49186008013112 (T) = (0 -118.57305913949928) / Math.Sqrt((69.43201713600281 / (299)) + (6.400239429652134 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.35233070963210367 = (183.0765498733869 - 118.57305913949928) / 183.0765498733869 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Floor - Duration of single invocation 30.50 μs 28.30 μs 0.93 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Floor

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.302922110884666 < 29.49026550587723.
IsChangePoint: Marked as a change because one of 11/24/2022 9:42:44 PM, 1/5/2023 9:29:47 PM, 1/18/2023 8:37:25 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 14.523614159905145 (T) = (0 -27920.0270435899) / Math.Sqrt((1153830.6230709376 / (299)) + (297510.22045710875 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08378620509399022 = (30473.266391338377 - 27920.0270435899) / 30473.266391338377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Load - Duration of single invocation 119.19 μs 107.33 μs 0.90 0.01 False
Load - Duration of single invocation 374.14 μs 326.55 μs 0.87 0.01 False
Load - Duration of single invocation 435.80 μs 383.77 μs 0.88 0.00 False
Load - Duration of single invocation 439.54 μs 360.19 μs 0.82 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "simple.xml")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 107.3313427579365 < 113.07902769704255.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 38.43787960375567 (T) = (0 -107765.67902594084) / Math.Sqrt((2575585.27466619 / (299)) + (1046084.5717113947 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1030318983292975 = (120144.3828662528 - 107765.67902594084) / 120144.3828662528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 326.5518337983632 < 355.2068072316113.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 78.08692893436063 (T) = (0 -323809.63673463813) / Math.Sqrt((19151346.511683084 / (299)) + (4568784.2518776115 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1430791048498071 = (377875.7625905293 - 323809.63673463813) / 377875.7625905293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "names.xml")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 383.76944381533104 < 414.2353608165923.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 53.551253387388286 (T) = (0 -383272.38295900205) / Math.Sqrt((26649961.06349197 / (299)) + (12835597.767622162 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13538759787994314 = (443288.0930451681 - 383272.38295900205) / 443288.0930451681 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "deep.xml")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 360.18530275974024 < 418.27519518634256.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 114.65181079344234 (T) = (0 -356355.50840564736) / Math.Sqrt((19404574.778834734 / (299)) + (5781690.658537996 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19822641319420234 = (444459.027174167 - 356355.50840564736) / 444459.027174167 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GenericClassWithSTringGenericInstanceMethod - Duration of single invocation 86.99 μs 76.15 μs 0.88 0.06 False
StaticDelegate - Duration of single invocation 217.60 μs 174.03 μs 0.80 0.01 False
GenericClassGenericStaticField - Duration of single invocation 71.94 μs 54.39 μs 0.76 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.GenericClassWithSTringGenericInstanceMethod

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.1547581881533 < 82.67101403135493.
IsChangePoint: Marked as a change because one of 11/16/2022 9:45:14 AM, 11/17/2022 4:58:45 AM, 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 52.46157214976859 (T) = (0 -76173.88137705815) / Math.Sqrt((11517330.255227 / (299)) + (3259.621304841789 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11947604096973061 = (86509.72025900269 - 76173.88137705815) / 86509.72025900269 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.StaticDelegate

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.0260833333333 < 206.646317109375.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 19.048489803666072 (T) = (0 -174135.2914580327) / Math.Sqrt((465614372.21451116 / (299)) + (25102.531207645363 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12018753846508089 = (197923.1928066086 - 174135.2914580327) / 197923.1928066086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericClassGenericStaticField

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.385562500000006 < 68.35307423468595.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 40.328526180518075 (T) = (0 -54406.09066049254) / Math.Sqrt((37589799.32006203 / (299)) + (2818.1440090191463 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.20829152362492598 = (68719.85368856593 - 54406.09066049254) / 68719.85368856593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 192.42 μs 174.68 μs 0.91 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Fib.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.68267286324783 < 182.8142523211354.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 24.91858153386432 (T) = (0 -174642.88138753606) / Math.Sqrt((62359884.88142825 / (299)) + (3608.0904640786634 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06122016322464647 = (186031.777149606 - 174642.88138753606) / 186031.777149606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 51.26 μs 27.52 μs 0.54 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.52347211081402 < 48.700489272020945.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 165.52293629814517 (T) = (0 -27665.731669272358) / Math.Sqrt((2892749.249910172 / (299)) + (158057.50104961017 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4812518565206044 = (53331.721794145036 - 27665.731669272358) / 53331.721794145036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IFooObjIsIFoo - Duration of single invocation 261.35 μs 239.79 μs 0.92 0.05 False
IFooObjIsIFooInterAlia - Duration of single invocation 261.04 μs 239.90 μs 0.92 0.05 False
IFooObjIsDescendantOfIFoo - Duration of single invocation 260.90 μs 239.90 μs 0.92 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.IFooObjIsIFoo

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 239.79160846445222 < 248.00175679054846.
IsChangePoint: Marked as a change because one of 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 19.088099501281526 (T) = (0 -239862.41036600585) / Math.Sqrt((164309195.01884982 / (299)) + (31408.259605923864 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.055842595234131695 = (254049.17565147605 - 239862.41036600585) / 254049.17565147605 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.IFooObjIsIFooInterAlia

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 239.90058820346317 < 248.0654849404762.
IsChangePoint: Marked as a change because one of 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 17.597588091000084 (T) = (0 -239777.47586343976) / Math.Sqrt((193506249.68983003 / (299)) + (4949.350592955521 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.055768148951234234 = (253939.19469790923 - 239777.47586343976) / 253939.19469790923 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.IFooObjIsDescendantOfIFoo

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 239.90492462121213 < 247.9898958107639.
IsChangePoint: Marked as a change because one of 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 21.23541014291107 (T) = (0 -239832.09592666847) / Math.Sqrt((120706257.03751652 / (299)) + (5772.54113636188 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05329418008931656 = (253333.2856761093 - 239832.09592666847) / 253333.2856761093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteViaIndexer2 - Duration of single invocation 1.58 μs 1.35 μs 0.86 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.WriteViaIndexer2(length: 1024)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3537403008562257 < 1.496897823077141.
IsChangePoint: Marked as a change because one of 12/20/2022 9:29:18 AM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 17.314588149289705 (T) = (0 -1358.3124033901895) / Math.Sqrt((11783.103524199294 / (299)) + (222.43369968346548 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0896127267404095 = (1492.0160279996317 - 1358.3124033901895) / 1492.0160279996317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Benchstone.BenchI.BubbleSort2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 57.40 ms 52.43 ms 0.91 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort2*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.BubbleSort2.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.43074343333333 < 54.528359202628195.
IsChangePoint: Marked as a change because one of 11/25/2022 11:21:20 AM, 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 50.720316899076124 (T) = (0 -52362107.63890276) / Math.Sqrt((2496082142149.7207 / (299)) + (820103728.4651254 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08163988371728495 = (57016966.12310547 - 52362107.63890276) / 57016966.12310547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CeilingDoubleBenchmark - Duration of single invocation 13.04 ns 11.95 ns 0.92 0.01 False
FloorFloatBenchmark - Duration of single invocation 44.34 ns 39.28 ns 0.89 0.03 False
CeilingFloatBenchmark - Duration of single invocation 42.97 ns 38.24 ns 0.89 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128.CeilingDoubleBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.95370851973763 < 12.276342801230195.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 101.36890370393903 (T) = (0 -11.967435034467739) / Math.Sqrt((0.011342201884882646 / (299)) + (0.0005540958099398984 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0737288153365493 = (12.920012230344788 - 11.967435034467739) / 12.920012230344788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128.FloorFloatBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.279279310375166 < 42.08241919018596.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/6/2022 2:12:43 AM, 12/20/2022 9:29:18 AM, 1/9/2023 10:41:51 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 14.751233036116382 (T) = (0 -39.97859237811766) / Math.Sqrt((1.1040966423730085 / (299)) + (0.4691010595989115 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07358289723792202 = (43.153987830020604 - 39.97859237811766) / 43.153987830020604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128.CeilingFloatBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.24490518827631 < 40.68672019790403.
IsChangePoint: Marked as a change because one of 11/20/2022 3:33:42 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 21.45496075778677 (T) = (0 -38.822612248209964) / Math.Sqrt((1.9710630079382263 / (299)) + (0.4145429249165655 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1041741766879359 = (43.33723279451163 - 38.822612248209964) / 43.33723279451163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PopCount_uint - Duration of single invocation 1.10 μs 667.95 ns 0.61 0.02 False
Log2_uint - Duration of single invocation 1.54 μs 791.73 ns 0.52 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BitOperations.PopCount_uint

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 667.9455427601066 < 1.0477823581585053.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/14/2022 12:26:48 AM, 12/20/2022 9:29:18 AM, 1/9/2023 10:41:51 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 30.947050354609154 (T) = (0 -667.8427007369665) / Math.Sqrt((34082.06038468536 / (299)) + (0.24024169913078613 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3310061851443886 = (998.2793351850446 - 667.8427007369665) / 998.2793351850446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.Log2_uint

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 791.7301135142109 < 1.4599769414255743.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/14/2022 12:26:48 AM, 12/20/2022 9:29:18 AM, 1/9/2023 10:41:51 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 30.991502383961123 (T) = (0 -791.921504934494) / Math.Sqrt((100664.94629265474 / (299)) + (0.1254806952968895 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4179541192693329 = (1360.582612388496 - 791.921504934494) / 1360.582612388496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AppendAllLines - Duration of single invocation 18.99 μs 17.30 μs 0.91 0.19 False
AppendAllText - Duration of single invocation 67.52 μs 47.19 μs 0.70 0.04 False
WriteAllText - Duration of single invocation 136.84 μs 75.07 μs 0.55 0.03 False
WriteAllText - Duration of single invocation 1.15 ms 530.18 μs 0.46 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_File.AppendAllLines

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.304121866666666 < 18.386630453411172.
IsChangePoint: Marked as a change because one of 11/19/2022 6:31:00 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 32.20587880392482 (T) = (0 -17371.813730383903) / Math.Sqrt((389428.25176840543 / (299)) + (29938.515403353133 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10522940022014027 = (19414.8240170809 - 17371.813730383903) / 19414.8240170809 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.AppendAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.19125580357142 < 63.82941253904532.
IsChangePoint: Marked as a change because one of 11/16/2022 11:33:39 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 216.68977392455142 (T) = (0 -47208.60319512987) / Math.Sqrt((2028529.7332155588 / (299)) + (18650.1645288465 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2971010767488495 = (67162.71946579995 - 47208.60319512987) / 67162.71946579995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 75.06811399948737 < 130.0506837781444.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 174.76246586389138 (T) = (0 -75123.11865490332) / Math.Sqrt((18960873.77799939 / (299)) + (881526.9307602255 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.46848539341470197 = (141337.82538457384 - 75123.11865490332) / 141337.82538457384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 530.1795808189655 < 1.0947755311123513.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 214.61437185288742 (T) = (0 -531486.5434376543) / Math.Sqrt((1779160192.5622032 / (299)) + (39254339.05871515 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5547315442263755 = (1193631.699138066 - 531486.5434376543) / 1193631.699138066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteString - Duration of single invocation 395.72 ms 329.93 ms 0.83 0.00 False
WriteCharArray - Duration of single invocation 388.58 ms 321.53 ms 0.83 0.00 False
WriteCharArray - Duration of single invocation 10.18 secs 6.89 secs 0.68 0.01 False
WritePartialCharArray - Duration of single invocation 518.60 ms 454.88 ms 0.88 0.02 False
WritePartialCharArray - Duration of single invocation 10.32 secs 7.02 secs 0.68 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 2)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 329.9281500833333 < 375.5695317196429.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 121.06645292735912 (T) = (0 -330230766.13911086) / Math.Sqrt((15043950422820.975 / (299)) + (2707788981789.765 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1664019369011525 = (396151071.7905211 - 330230766.13911086) / 396151071.7905211 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 2)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 321.5258017857143 < 369.1552166066666.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 164.95080667200438 (T) = (0 -322256942.1146187) / Math.Sqrt((12691265044088.977 / (299)) + (1328202006755.4695 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1713459922028309 = (388892033.4450347 - 322256942.1146187) / 388892033.4450347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.8887897495 < 9.654868668281921.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 288.7996894718736 (T) = (0 -6892787587.858443) / Math.Sqrt((37209724496217024 / (299)) + (81281936038335.45 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3248182205831613 = (10208787911.622576 - 6892787587.858443) / 10208787911.622576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 2)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 454.8830334 < 493.0995941157968.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 47.4399897410501 (T) = (0 -452944934.9404762) / Math.Sqrt((414625883986627.25 / (299)) + (629874028164.3379 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11178793181441099 = (509951340.63618106 - 452944934.9404762) / 509951340.63618106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.016014145214286 < 9.797908941389286.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 284.72133728123475 (T) = (0 -7028483296.446786) / Math.Sqrt((37397255308215656 / (299)) + (115349993384211.17 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3204934205888282 = (10343510290.27171 - 7028483296.446786) / 10343510290.27171 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 5.11 ms 4.56 ms 0.89 0.00 False
Parse - Duration of single invocation 14.69 μs 13.29 μs 0.90 0.00 False
Parse - Duration of single invocation 4.83 ms 4.26 ms 0.88 0.00 False
Parse - Duration of single invocation 7.07 μs 6.69 μs 0.95 0.00 False
Parse - Duration of single invocation 5.16 μs 4.60 μs 0.89 0.01 False
Parse - Duration of single invocation 3.88 μs 3.48 μs 0.90 0.01 False
Parse - Duration of single invocation 5.60 μs 4.93 μs 0.88 0.00 False
Parse - Duration of single invocation 14.17 μs 12.95 μs 0.91 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.559337666294643 < 4.860022149568596.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 171.51036081647896 (T) = (0 -4545988.166257701) / Math.Sqrt((891128395.8104055 / (299)) + (79743305.68436006 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10767562698595086 = (5094546.673540348 - 4545988.166257701) / 5094546.673540348 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.29295430237316 < 13.898011756056587.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 72.74519777227958 (T) = (0 -13355.935382455522) / Math.Sqrt((29917.40979624826 / (299)) + (3443.2441913716007 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0996668161893419 = (14834.436431551436 - 13355.935382455522) / 14834.436431551436 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.264233795758928 < 4.582999803766741.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 184.57679917112625 (T) = (0 -4263440.469501331) / Math.Sqrt((1075637298.746407 / (299)) + (45952800.501127295 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10771364017779682 = (4778107.860295952 - 4263440.469501331) / 4778107.860295952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.692917747352797 < 6.730884057740626.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 48.65677454452131 (T) = (0 -6711.471840290148) / Math.Sqrt((14101.163584517088 / (299)) + (631.2199710384056 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06901141665643476 = (7208.973300388363 - 6711.471840290148) / 7208.973300388363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.595700366024132 < 4.906080613261456.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 133.29570678980198 (T) = (0 -4614.069485726968) / Math.Sqrt((1227.5738226132046 / (299)) + (140.581811436381 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10611439804456647 = (5161.812065921397 - 4614.069485726968) / 5161.812065921397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4790721979335437 < 3.626202420603381.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 48.91975218021137 (T) = (0 -3468.803805748673) / Math.Sqrt((1366.882487365246 / (299)) + (641.4030780482783 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10058307040080942 = (3856.725053301459 - 3468.803805748673) / 3856.725053301459 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.926935683041345 < 5.306468462508533.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 190.31450970577743 (T) = (0 -4934.162888920343) / Math.Sqrt((1067.6033960529105 / (299)) + (88.31650592442554 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.116110921503503 = (5582.332680604445 - 4934.162888920343) / 5582.332680604445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.947200445318972 < 13.510337201832716.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/18/2023 3:58:17 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 94.8471366044228 (T) = (0 -12983.820659180057) / Math.Sqrt((28398.933979024187 / (299)) + (1441.3962181278284 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.09895495668086164 = (14409.735401631146 - 12983.820659180057) / 14409.735401631146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFullPathWithRedundantSegments - Duration of single invocation 5.37 μs 4.06 μs 0.76 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFullPathWithRedundantSegments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.058299311755953 < 5.0736932773002845.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 73.77351461585438 (T) = (0 -4019.330979346366) / Math.Sqrt((13294.242434959908 / (299)) + (3450.868850655781 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.257814985033087 = (5415.537767931827 - 4019.330979346366) / 5415.537767931827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 97.53 μs 92.16 μs 0.95 0.00 False
ParseThenWrite - Duration of single invocation 11.34 μs 10.67 μs 0.94 0.00 False
ParseThenWrite - Duration of single invocation 10.71 μs 10.08 μs 0.94 0.00 False
ParseThenWrite - Duration of single invocation 9.70 ms 9.14 ms 0.94 0.00 False
ParseThenWrite - Duration of single invocation 10.19 ms 9.64 ms 0.95 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.16479841911764 < 92.643506813848.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 13.38050555499127 (T) = (0 -92256.37941305753) / Math.Sqrt((153281.40513168092 / (299)) + (1477162.9942455762 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05055799169005775 = (97169.0515119284 - 92256.37941305753) / 97169.0515119284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.67398788483119 < 10.779071736326497.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 63.89317775590432 (T) = (0 -10658.510907611128) / Math.Sqrt((2635.4986032867773 / (299)) + (1225.7289539751253 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06167957070475004 = (11359.13764087656 - 10658.510907611128) / 11359.13764087656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.075465293128856 < 10.159511484400154.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 11.617088900985596 (T) = (0 -10110.122652997012) / Math.Sqrt((2526.4218461675605 / (299)) + (23868.258601548212 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05089922739120053 = (10652.317377434276 - 10110.122652997012) / 10652.317377434276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.140420268229166 < 9.20450534031622.
IsChangePoint: Marked as a change because one of 1/8/2023 3:40:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 93.8501306269068 (T) = (0 -9141582.974524954) / Math.Sqrt((2139708065.4189708 / (299)) + (253106112.7718575 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05337632858213308 = (9657040.332440194 - 9141582.974524954) / 9657040.332440194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.644749838942309 < 9.688085989479168.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 93.64933380606114 (T) = (0 -9622798.882841118) / Math.Sqrt((1322840801.492306 / (299)) + (366641269.0291676 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05642460349245733 = (10198229.96493762 - 9622798.882841118) / 10198229.96493762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 1.10 μs 905.52 ns 0.83 0.01 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 11.03 μs 8.91 μs 0.81 0.02 False
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 10.94 μs 8.83 μs 0.81 0.01 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 1.19 μs 978.61 ns 0.82 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 905.5160735054245 < 1.0403795383912688.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 164.66026627133934 (T) = (0 -906.0057392732804) / Math.Sqrt((309.1267682112416 / (299)) + (3.120996990545318 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17260778975407903 = (1095.0136199662716 - 906.0057392732804) / 1095.0136199662716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.908355958662703 < 10.484998593352062.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 168.67104389088288 (T) = (0 -8905.702663687664) / Math.Sqrt((15583.11833692798 / (299)) + (1087.2678656558187 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18877503949471552 = (10978.092511034922 - 8905.702663687664) / 10978.092511034922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.834218855932201 < 10.389040290438327.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 206.87565007492887 (T) = (0 -8828.998785265496) / Math.Sqrt((14755.571707243425 / (299)) + (544.8493800993684 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18896982272438376 = (10886.153231589427 - 8828.998785265496) / 10886.153231589427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 978.6137054126041 < 1.131517947194717.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 264.7995797588546 (T) = (0 -977.6391137762213) / Math.Sqrt((159.8997627902346 / (299)) + (1.0313649720307936 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17677623031519837 = (1187.5739619988653 - 977.6391137762213) / 1187.5739619988653 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 39.64 μs 35.34 μs 0.89 0.00 False
Parse - Duration of single invocation 4.82 ms 4.27 ms 0.88 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfStrings)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.344699788374726 < 37.646198783548066.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 27.321298617549505 (T) = (0 -35481.87226217025) / Math.Sqrt((50147.75440514476 / (299)) + (212470.61586341713 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.09704867587629164 = (39295.44297042204 - 35481.87226217025) / 39295.44297042204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.266591674479167 < 4.580230852691592.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 56.45509828181101 (T) = (0 -4267706.844215159) / Math.Sqrt((1184137102.1932414 / (299)) + (852854766.3598708 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10667817045150807 = (4777345.30048613 - 4267706.844215159) / 4777345.30048613 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_DecodingRequired - Duration of single invocation 389.33 ns 330.71 ns 0.85 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 330.7055000834502 < 373.56074210631414.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 62.65712016377232 (T) = (0 -333.9592502960041) / Math.Sqrt((96.90342908751887 / (299)) + (8.458650724978975 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.16398780817196443 = (399.4669617984449 - 333.9592502960041) / 399.4669617984449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 3.99 ms 3.60 ms 0.90 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: ObjectProperties)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6000125607142857 < 3.794544774497768.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 23.61290157177206 (T) = (0 -3617191.7839210792) / Math.Sqrt((565606570.185438 / (299)) + (2333691202.8942227 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08717982764976026 = (3962655.3986070324 - 3617191.7839210792) / 3962655.3986070324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Benchstone.BenchI.EightQueens

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.92 μs 1.75 μs 0.91 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.EightQueens.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7454122968155454 < 1.7868204669032077.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 34.32949408029408 (T) = (0 -1738.8120735203106) / Math.Sqrt((1908.4723384104416 / (299)) + (59.16109040272429 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06341424942342438 = (1856.543378382463 - 1738.8120735203106) / 1856.543378382463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in BilinearTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Interpol_Vector - Duration of single invocation 117.84 μs 95.91 μs 0.81 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BilinearTest*'

Payloads

Baseline
Compare

Histogram

BilinearTest.Interpol_Vector

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.9109225609756 < 112.01634019982993.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 154.36218351395067 (T) = (0 -96184.14079280781) / Math.Sqrt((570057.2741261825 / (299)) + (210582.41141688047 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18886827883683763 = (118580.17419769974 - 96184.14079280781) / 118580.17419769974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString_NonFlags_Large - Duration of single invocation 41.19 ns 38.29 ns 0.93 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_NonFlags_Large(value: Format)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.292126769841495 < 39.05162171261403.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 19.674195551491767 (T) = (0 -38.581221038675416) / Math.Sqrt((0.3740455632104461 / (99)) + (0.12751056751391313 / (11))) is greater than 1.9821734833073787 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (11) - 2, .975) and 0.05946152433137472 = (41.0203538045035 - 38.581221038675416) / 41.0203538045035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Benchstone.BenchI.Array1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 7.28 ms 6.89 ms 0.95 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Array1*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Array1.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.890913956730769 < 6.912983914314331.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 47.548087846421595 (T) = (0 -6894142.722527473) / Math.Sqrt((17903797931.420567 / (299)) + (12483955.220929462 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.051118589414613266 = (7265547.249233514 - 6894142.722527473) / 7265547.249233514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Benchstone.BenchI.Array2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.21 secs 1.06 secs 0.88 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Array2*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Array2.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0607136914615385 < 1.141888629496978.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 35.184553601916626 (T) = (0 -1069060292.8468198) / Math.Sqrt((1127010880775659.5 / (299)) + (66397936997517.336 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.09342969885963079 = (1179235952.801515 - 1069060292.8468198) / 1179235952.801515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NestedScopes_TwoMessages - Duration of single invocation 5.76 μs 5.07 μs 0.88 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: True)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.065183298404232 < 5.542037705964277.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 23.278391304362003 (T) = (0 -5153.690271476113) / Math.Sqrt((36098.53208837598 / (299)) + (9769.693282907181 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12546753808591615 = (5893.080584105777 - 5153.690271476113) / 5893.080584105777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Remainder - Duration of single invocation 6.36 ms 5.67 ms 0.89 0.01 False
Divide - Duration of single invocation 6.35 ms 5.67 ms 0.89 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 65536,32768 bits)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.668859547619046 < 6.032596092994791.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/10/2022 11:25:25 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 76.76281710496423 (T) = (0 -5669758.704077868) / Math.Sqrt((23411062089.63059 / (299)) + (7201046.713948935 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1073827598159928 = (6351836.429810704 - 5669758.704077868) / 6351836.429810704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 65536,32768 bits)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.667622875000001 < 6.03364102751736.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/10/2022 11:25:25 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 76.1080906953032 (T) = (0 -5669199.929354674) / Math.Sqrt((23638163818.009743 / (299)) + (13735223.984454716 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10738583315336137 = (6351232.301614039 - 5669199.929354674) / 6351232.301614039 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in Benchstone.MDBenchF.MDInvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 7.82 ms 7.39 ms 0.95 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchF.MDInvMt*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDInvMt.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.392468696428572 < 7.421147698660714.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 35.68056737433051 (T) = (0 -7396451.504479548) / Math.Sqrt((67947767323.52953 / (299)) + (210673289.68592116 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07039307794267083 = (7956536.605934832 - 7396451.504479548) / 7956536.605934832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddBenchmark - Duration of single invocation 27.16 ns 20.72 ns 0.76 0.01 False
LessThanBenchmark - Duration of single invocation 28.47 ns 22.28 ns 0.78 0.03 False
GreaterThanBenchmark - Duration of single invocation 35.26 ns 22.26 ns 0.63 0.01 False
GreaterThanOrEqualBenchmark - Duration of single invocation 36.55 ns 22.74 ns 0.62 0.02 False
LessThanAnyBenchmark - Duration of single invocation 22.38 ns 7.29 ns 0.33 0.04 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 4.72 ns 2.97 ns 0.63 0.10 False
GreaterThanAnyBenchmark - Duration of single invocation 4.78 ns 2.68 ns 0.56 0.09 False
UnaryNegateOperatorBenchmark - Duration of single invocation 26.83 ns 20.34 ns 0.76 0.03 False
DivisionOperatorBenchmark - Duration of single invocation 30.92 ns 23.49 ns 0.76 0.02 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 20.95 ns 8.12 ns 0.39 0.04 False
MultiplyBenchmark - Duration of single invocation 27.10 ns 20.87 ns 0.77 0.01 False
EqualsAllBenchmark - Duration of single invocation 4.61 ns 2.72 ns 0.59 0.09 False
EqualsAnyBenchmark - Duration of single invocation 21.48 ns 8.08 ns 0.38 0.01 False
LessThanAllBenchmark - Duration of single invocation 5.28 ns 2.70 ns 0.51 0.10 False
DotBenchmark - Duration of single invocation 31.23 ns 6.42 ns 0.21 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 23.24 ns 7.66 ns 0.33 0.05 False
EqualsStaticBenchmark - Duration of single invocation 28.40 ns 22.30 ns 0.79 0.02 False
SubtractionOperatorBenchmark - Duration of single invocation 27.08 ns 20.92 ns 0.77 0.01 False
AbsBenchmark - Duration of single invocation 29.31 ns 19.94 ns 0.68 0.02 False
AddOperatorBenchmark - Duration of single invocation 27.14 ns 21.01 ns 0.77 0.02 False
MaxBenchmark - Duration of single invocation 29.26 ns 23.16 ns 0.79 0.02 False
InequalityOperatorBenchmark - Duration of single invocation 5.46 ns 2.67 ns 0.49 0.07 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 25.35 ns 8.57 ns 0.34 0.04 False
SubtractBenchmark - Duration of single invocation 27.55 ns 20.99 ns 0.76 0.03 False
SumBenchmark - Duration of single invocation 17.56 ns 3.88 ns 0.22 0.04 False
DivideBenchmark - Duration of single invocation 30.92 ns 23.76 ns 0.77 0.03 False
EqualityOperatorBenchmark - Duration of single invocation 5.25 ns 2.67 ns 0.51 0.09 False
NegateBenchmark - Duration of single invocation 26.71 ns 20.27 ns 0.76 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.717021879785232 < 25.79962518991467.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 77.12927939651674 (T) = (0 -20.936738341003778) / Math.Sqrt((0.07469068834377457 / (299)) + (0.0708728595641264 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.23158467979645336 = (27.246643566994234 - 20.936738341003778) / 27.246643566994234 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.28423461067603 < 27.039180963635932.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 353.49991305937203 (T) = (0 -22.306377560796594) / Math.Sqrt((0.06612421134657144 / (299)) + (0.0008700863097108498 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.21544060960487257 = (28.43172592652602 - 22.306377560796594) / 28.43172592652602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.255540652128424 < 33.46978615516703.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 167.01053379966115 (T) = (0 -22.36403960801932) / Math.Sqrt((1.209459476749283 / (299)) + (0.020644202689268348 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3649465789998752 = (35.21599737672293 - 22.36403960801932) / 35.21599737672293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.74000019486144 < 34.76617335990386.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 479.7638305460037 (T) = (0 -22.747036991309997) / Math.Sqrt((0.22863311419672486 / (299)) + (0.00048010229028323575 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.37485917165029703 = (36.387060258661165 - 22.747036991309997) / 36.387060258661165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.289473341810653 < 21.28645714968866.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 208.05840788702235 (T) = (0 -7.343094348101306) / Math.Sqrt((1.2952496792216734 / (299)) + (0.008060176226830654 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6684818216733714 = (22.149899547488815 - 7.343094348101306) / 22.149899547488815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9669040621293417 < 4.4693486629646095.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 50.70811028431582 (T) = (0 -2.9564715756865185) / Math.Sqrt((0.3324857660767678 / (299)) + (0.0017001892091339829 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.37903694266707705 = (4.761107026857214 - 2.9564715756865185) / 4.761107026857214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6798519405232404 < 4.542913046622961.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/6/2022 2:12:43 AM, 1/3/2023 6:29:42 AM, 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 37.98336927865938 (T) = (0 -2.72313099313364) / Math.Sqrt((0.5832106920136068 / (299)) + (0.020560234598377126 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.46295993329767443 = (5.070629105673482 - 2.72313099313364) / 5.070629105673482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.338213325725082 < 25.391339198421583.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 121.66395289936158 (T) = (0 -20.438756472553308) / Math.Sqrt((0.12560253100945712 / (299)) + (0.026013413089686244 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2390424832526034 = (26.859260895293907 - 20.438756472553308) / 26.859260895293907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.49379839532054 < 29.36338009258876.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 104.68025521290978 (T) = (0 -23.523312147392073) / Math.Sqrt((0.36614318159933723 / (299)) + (0.05123476515092913 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2544556417563447 = (31.551861250493555 - 23.523312147392073) / 31.551861250493555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.123510031965996 < 19.937028685854564.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 268.0393499644532 (T) = (0 -8.124161752807277) / Math.Sqrt((0.6816687348235856 / (299)) + (0.0008288113464333153 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6155544235801614 = (21.132150429363204 - 8.124161752807277) / 21.132150429363204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.8738935434578 < 25.865747683635217.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 83.35266509614979 (T) = (0 -21.08124245254308) / Math.Sqrt((0.04843659991309174 / (299)) + (0.05627332562095565 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.22314523549353107 = (27.13665850519159 - 21.08124245254308) / 27.13665850519159 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.717656794414128 < 4.367153308551497.
IsChangePoint: Marked as a change because one of 12/20/2022 4:30:45 AM, 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 51.70577799032352 (T) = (0 -2.710723454691395) / Math.Sqrt((0.5167954773590487 / (299)) + (0.0003586562519082996 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4445841868509276 = (4.880529849019338 - 2.710723454691395) / 4.880529849019338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.083651981573313 < 20.417747537377224.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 346.9337124672528 (T) = (0 -8.110342268191483) / Math.Sqrt((0.29896901562532874 / (299)) + (0.006565511962152935 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6309057963881247 = (21.973637593940634 - 8.110342268191483) / 21.973637593940634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7028014392512394 < 5.0888119603281075.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 43.93764991018979 (T) = (0 -2.700086758269703) / Math.Sqrt((0.9164624418975995 / (299)) + (2.9284217724491036E-06 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4739466724525802 = (5.132724415712987 - 2.700086758269703) / 5.132724415712987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.422642281713727 < 29.658057066833138.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 522.100962270988 (T) = (0 -6.395081094590947) / Math.Sqrt((0.6942298089645366 / (299)) + (0.00019467660321445922 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.7979338672229096 = (31.648455912429874 - 6.395081094590947) / 31.648455912429874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.657086378397042 < 22.09186126047534.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 343.38808576569625 (T) = (0 -7.652341574139727) / Math.Sqrt((0.5610647613329447 / (299)) + (8.798594329521195E-05 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.66078515693222 = (22.55898210388947 - 7.652341574139727) / 22.55898210388947 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.30231209842601 < 26.981953927843783.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 169.6527395119143 (T) = (0 -22.30860802457502) / Math.Sqrt((0.32914121864202095 / (299)) + (0.0023424429471172524 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.21608115142294218 = (28.457802826234918 - 22.30860802457502) / 28.457802826234918 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.922069608491913 < 25.77480095010585.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 118.90786392273932 (T) = (0 -20.961843068608832) / Math.Sqrt((0.050939732652332544 / (299)) + (0.0282926939473944 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.22902756071462982 = (27.18883581368847 - 20.961843068608832) / 27.18883581368847 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.944994497753047 < 27.844427097616432.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 174.76115277722928 (T) = (0 -19.942797343315327) / Math.Sqrt((0.05545150062104179 / (299)) + (0.028470150001395383 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3157779864605694 = (29.146676003820296 - 19.942797343315327) / 29.146676003820296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.01464920262241 < 25.768531364993752.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 6.90468667623674 (T) = (0 -21.873670011715603) / Math.Sqrt((0.08330951874405941 / (299)) + (6.509389123977336 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19541947003103505 = (27.186427208920076 - 21.873670011715603) / 27.186427208920076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.160634235805826 < 27.794676049832805.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 222.12354517503195 (T) = (0 -23.19892523942732) / Math.Sqrt((0.061206336667418385 / (299)) + (0.006182113342687455 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.20956148086283308 = (29.349436645307957 - 23.19892523942732) / 29.349436645307957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.671865921227829 < 5.188620307162096.
IsChangePoint: Marked as a change because one of 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 124.97074043526645 (T) = (0 -2.673083806312228) / Math.Sqrt((0.1159245375019414 / (299)) + (1.958865461188593E-05 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4798879673623777 = (5.139438502809348 - 2.673083806312228) / 5.139438502809348 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.565396634351726 < 23.960569969809427.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 118.06720570964576 (T) = (0 -8.603462161707647) / Math.Sqrt((1.3245495615012064 / (299)) + (0.14206702177849867 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6437934376098116 = (24.1530142060758 - 8.603462161707647) / 24.1530142060758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.989605502732466 < 26.097738496541815.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 9.77313049268011 (T) = (0 -21.647155963537163) / Math.Sqrt((0.2175824940456889 / (299)) + (3.720835366601782 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.20814616434016192 = (27.337312757346137 - 21.647155963537163) / 27.337312757346137 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SumBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.8755404957137443 < 16.810369221106185.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 253.01456340209344 (T) = (0 -3.859273054918863) / Math.Sqrt((0.8760005366319981 / (299)) + (9.814050868583286E-05 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.7804126040723542 = (17.57511189845567 - 3.859273054918863) / 17.57511189845567 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.75685700204279 < 29.461570081108825.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 116.22939650375939 (T) = (0 -23.441631704090945) / Math.Sqrt((0.3253861584391036 / (299)) + (0.038289642595705595 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.251022832008952 = (31.298192663158893 - 23.441631704090945) / 31.298192663158893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6713210984564584 < 4.991404218008908.
IsChangePoint: Marked as a change because one of 1/18/2023 11:30:09 AM, 1/18/2023 8:37:25 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 38.617761911792805 (T) = (0 -2.7344669685983893) / Math.Sqrt((0.5080574162927582 / (299)) + (0.01970806627809132 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4548652176033539 = (5.016130059756049 - 2.7344669685983893) / 5.016130059756049 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.268850481530638 < 25.289900157204947.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 133.55888370005394 (T) = (0 -20.371260740735934) / Math.Sqrt((0.18103470753852677 / (299)) + (0.01845798491902614 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.23855590000578927 = (26.753455363159052 - 20.371260740735934) / 26.753455363159052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 31, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FixedTimeEquals_256Bit_AllBitsDifferent - Duration of single invocation 61.17 ns 49.97 ns 0.82 0.03 False
FixedTimeEquals_256Bit_LastBitDifferent - Duration of single invocation 61.32 ns 50.84 ns 0.83 0.03 False
FixedTimeEquals_256Bit_CascadingErrors - Duration of single invocation 59.97 ns 49.78 ns 0.83 0.02 False
FixedTimeEquals_256Bit_FirstBitDifferent - Duration of single invocation 60.35 ns 49.90 ns 0.83 0.02 False
FixedTimeEquals_256Bit_VersusZero - Duration of single invocation 60.23 ns 51.05 ns 0.85 0.04 False
FixedTimeEquals_256Bit_SameReference - Duration of single invocation 59.66 ns 50.73 ns 0.85 0.03 False
FixedTimeEquals_256Bit_Equal - Duration of single invocation 61.63 ns 50.27 ns 0.82 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.97225567085008 < 57.84719514271264.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/18/2022 12:43:31 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 21.835873581202577 (T) = (0 -50.39872739871598) / Math.Sqrt((25.78096048514848 / (299)) + (0.14804658522945707 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12033166323813052 = (57.292874248762644 - 50.39872739871598) / 57.292874248762644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.836396497120354 < 57.946970434670604.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/18/2022 12:43:31 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 15.234502732122762 (T) = (0 -51.13580350506448) / Math.Sqrt((25.293072350983067 / (299)) + (0.9047177308007074 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10848751996790065 = (57.35848308396458 - 51.13580350506448) / 57.35848308396458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.77908981238151 < 57.280237882689065.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/18/2022 12:43:31 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 19.30417114416462 (T) = (0 -50.27596255867804) / Math.Sqrt((25.217831132447994 / (299)) + (0.3851005826081203 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11711335151648232 = (56.944980020973354 - 50.27596255867804) / 56.944980020973354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.90190786275135 < 57.500200700955205.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/18/2022 12:43:31 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 21.274224082830354 (T) = (0 -50.19417335275814) / Math.Sqrt((26.160740309806442 / (299)) + (0.2602093484760351 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12380919272883754 = (57.28680663642721 - 50.19417335275814) / 57.28680663642721 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.05219017023434 < 57.403202479334276.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/18/2022 12:43:31 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 19.270457115962284 (T) = (0 -50.53216186881657) / Math.Sqrt((24.802404300371045 / (299)) + (0.4382902825821745 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11788080412906297 = (57.28495888690528 - 50.53216186881657) / 57.28495888690528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReference

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.72892373806126 < 57.21225657031119.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/18/2022 12:43:31 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 19.715095601984405 (T) = (0 -50.562331412089264) / Math.Sqrt((24.105214674699052 / (299)) + (0.4280813092131542 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11879495641381577 = (57.378622353679404 - 50.562331412089264) / 57.378622353679404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.268904089078994 < 58.019732292342866.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/18/2022 12:43:31 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 21.856116319290404 (T) = (0 -50.41759776282636) / Math.Sqrt((26.566148208906494 / (299)) + (0.11861230567281392 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12036367826700566 = (57.316411927485376 - 50.41759776282636) / 57.316411927485376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadToEnd - Duration of single invocation 494.01 ms 311.75 ms 0.63 0.02 False
ReadToEnd - Duration of single invocation 499.98 ms 310.89 ms 0.62 0.02 False
ReadToEnd - Duration of single invocation 492.81 ms 309.92 ms 0.63 0.02 False
ReadToEnd - Duration of single invocation 494.92 ms 311.46 ms 0.63 0.02 False
ReadToEnd - Duration of single invocation 492.11 ms 311.40 ms 0.63 0.00 False
ReadToEnd - Duration of single invocation 492.27 ms 311.39 ms 0.63 0.01 False
ReadToEnd - Duration of single invocation 493.43 ms 311.86 ms 0.63 0.02 False
ReadToEnd - Duration of single invocation 493.76 ms 312.29 ms 0.63 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 9, 32])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.7507359333333 < 468.8803129826923.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 130.56160443467778 (T) = (0 -312624497.937063) / Math.Sqrt((85135907419393.27 / (299)) + (20309538495813.008 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.37875204821729624 = (503220166.82706237 - 312624497.937063) / 503220166.82706237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 310.89338093333333 < 468.9418256166667.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 101.1583128310182 (T) = (0 -313821468.836297) / Math.Sqrt((92050987192002.39 / (299)) + (35435028025078.01 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.37716525552658375 = (503859926.92431027 - 313821468.836297) / 503859926.92431027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 309.9182062307692 < 468.36966682083323.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 212.20075131613544 (T) = (0 -311340951.9189144) / Math.Sqrt((91710434734652.45 / (299)) + (5635181400319.416 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.38150067852838515 = (503381234.40156907 - 311340951.9189144) / 503381234.40156907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.46329653333333 < 469.15205774230765.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 205.08527334769823 (T) = (0 -312752339.71212125) / Math.Sqrt((86236162295809.11 / (299)) + (6368902578011.261 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.379161298572345 = (503757802.13593143 - 312752339.71212125) / 503757802.13593143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.40340093333333 < 468.3146469533333.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 211.2910267792407 (T) = (0 -311519735.4486514) / Math.Sqrt((86658428304761.05 / (299)) + (5906833198532.17 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3814696449754853 = (503645023.91528493 - 311519735.4486514) / 503645023.91528493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.39067213333334 < 467.59638099732143.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 190.93915909899997 (T) = (0 -311570290.08924407) / Math.Sqrt((81891111000146.14 / (299)) + (8079701736638.734 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.38095732615537153 = (503309873.8641144 - 311570290.08924407) / 503309873.8641144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.85806706666665 < 468.61303138832415.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 207.77900613646503 (T) = (0 -312305775.1747253) / Math.Sqrt((95927863112013.34 / (299)) + (5792732696324.124 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.37982885285606 = (503579982.0952328 - 312305775.1747253) / 503579982.0952328 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 312.2925524 < 469.2329074872619.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 93.25104858312335 (T) = (0 -313860237.69525474) / Math.Sqrt((94909240522195.67 / (299)) + (42026583632966.16 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3767082512574615 = (503552691.5420473 - 313860237.69525474) / 503552691.5420473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Depth

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSpanEmptyLoop - Duration of single invocation 9.99 μs 8.73 μs 0.87 0.00 False
ReadSpanEmptyLoop - Duration of single invocation 89.21 μs 76.43 μs 0.86 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 10.22 μs 8.89 μs 0.87 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 64)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.725740910993304 < 9.510988328155111.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 113.52526338432291 (T) = (0 -8728.31029195479) / Math.Sqrt((10380.944164900546 / (299)) + (849.3610340216094 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1209622706064813 = (9929.392106954021 - 8728.31029195479) / 9929.392106954021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.42673312282228 < 84.72828701198084.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 144.69836839095498 (T) = (0 -76412.88623732362) / Math.Sqrt((1534724.6635124248 / (299)) + (10394.393539197592 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12863791912721137 = (87693.60971134483 - 76412.88623732362) / 87693.60971134483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 65)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.888161655871844 < 9.699946891794179.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 179.5567131791224 (T) = (0 -8896.327679168366) / Math.Sqrt((10769.396975431486 / (299)) + (137.5291755838907 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12326076475364864 = (10147.06234364956 - 8896.327679168366) / 10147.06234364956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7
Compare b6cd3003c05d9be521d6d1ffff9207255d177116
Diff Diff

Improvements in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FooObjIsNull - Duration of single invocation 217.50 μs 173.56 μs 0.80 0.05 False
IFooObjIsIFoo - Duration of single invocation 195.77 μs 173.94 μs 0.89 0.01 False
ObjFooIsObj - Duration of single invocation 218.11 μs 171.86 μs 0.79 0.05 False
IFooObjIsDescendantOfIFoo - Duration of single invocation 195.82 μs 174.06 μs 0.89 0.02 False
IFooFooIsIFoo - Duration of single invocation 217.58 μs 171.79 μs 0.79 0.06 False
ObjrefValueTypeObj - Duration of single invocation 565.53 μs 508.57 μs 0.90 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.FooObjIsNull

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 173.55528626373626 < 206.67520847236528.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 1/17/2023 2:19:41 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 17.54124543195121 (T) = (0 -173620.61571677387) / Math.Sqrt((517562325.82956177 / (299)) + (3563795.8552388162 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1265084244898809 = (198766.21662364565 - 173620.61571677387) / 198766.21662364565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFoo

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 173.94410486111113 < 185.9748021554129.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 63.81878699276763 (T) = (0 -173950.43829800934) / Math.Sqrt((30403928.49362532 / (299)) + (5223.449907179444 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10495623405278955 = (194348.5278777629 - 173950.43829800934) / 194348.5278777629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjFooIsObj

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.85760841503267 < 206.63914809194708.
IsChangePoint: Marked as a change because one of 12/15/2022 5:00:10 PM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 18.335905583078098 (T) = (0 -173406.42600014777) / Math.Sqrt((519075828.3683395 / (299)) + (2145077.5702845315 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1281131430906561 = (198886.3860327442 - 173406.42600014777) / 198886.3860327442 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsDescendantOfIFoo

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.06252003968254 < 185.9880949353966.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/20/2022 9:29:18 AM, 1/28/2023 4:01:39 AM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 24.199402488379416 (T) = (0 -174120.13492601438) / Math.Sqrt((226501949.03143227 / (299)) + (35631.27239582974 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10813674133035908 = (195231.8735337779 - 174120.13492601438) / 195231.8735337779 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooFooIsIFoo

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.78535122282605 < 206.68551110646084.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 1/17/2023 5:52:57 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 19.986746642135543 (T) = (0 -171767.74756249556) / Math.Sqrt((511397923.4021931 / (299)) + (1564711.163048803 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13672321115512828 = (198971.81272802842 - 171767.74756249556) / 198971.81272802842 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjrefValueTypeObj

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 508.56737641129024 < 537.3338197759272.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 1/17/2023 2:19:41 PM, 1/27/2023 11:43:56 PM, 1/31/2023 8:24:06 AM falls between 1/22/2023 4:26:49 PM and 1/31/2023 8:24:06 AM.
IsImprovementStdDev: Marked as improvement because 18.29477951986245 (T) = (0 -508164.49935407325) / Math.Sqrt((1193581108.7810268 / (299)) + (2803882.8135052575 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06906730060085078 = (545865.9897563565 - 508164.49935407325) / 545865.9897563565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@naricc
Copy link
Member

naricc commented Feb 1, 2023

@vargaz It looks like we also got some improvments out of either dotnet/runtime@e07f452
or
dotnet/runtime@ce66d01

Whatever is going on we got a lot more improvements than regressions out of this issue, and the regressions are pretty borderline, so I am going to close out the regressions: #12307

@naricc naricc closed this as completed Feb 1, 2023
@kotlarmilos kotlarmilos mentioned this issue Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@naricc