Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable wasm aot microbenchmark run #1902

Merged
merged 49 commits into from
Aug 4, 2021
Merged

Enable wasm aot microbenchmark run #1902

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
ff341e4
copy back the helix job folder for diagnosis
Lxiamail Jun 28, 2021
a1b4109
copy back the helix job folder for diagnosis
Lxiamail Jun 28, 2021
a4e9600
Merge branch 'alicial/hikeMicro-benchmark' of https://github.com/dotn…
Lxiamail Jun 28, 2021
ef302af
hack:setting WasmNativeWorkload=false for wasm run. should be removed…
Lxiamail Jun 29, 2021
d0709a5
fixed merge confict
Lxiamail Jun 29, 2021
62b5514
fix copyjob
Lxiamail Jun 29, 2021
b7c61b1
move copy helix job-* folder code to benchmarks_ci.py
Lxiamail Jun 29, 2021
fc72b62
update benchmarkdotnet version
Lxiamail Jun 29, 2021
9ebeee7
add more log info for diagnosis
Lxiamail Jun 30, 2021
da5b930
suppress an unnecessory exception
Lxiamail Jun 30, 2021
35e3c41
fix copy back job-* folder
Lxiamail Jul 1, 2021
e386e4e
fix copy job
Lxiamail Jul 1, 2021
4e0e027
correct helixuploadroot
Lxiamail Jul 1, 2021
d6abd59
add error handling
Lxiamail Jul 1, 2021
023ef60
fix copy path
Lxiamail Jul 2, 2021
6035f55
enabled binlog
Lxiamail Jul 9, 2021
ac4dcdc
rethrow
Lxiamail Jul 12, 2021
ab97a6b
add back copy job folder for diagnosis
Lxiamail Jul 14, 2021
0049153
dotnet workload install wasm-tools to work around missing workload error
Lxiamail Jul 15, 2021
8448dc7
run dotnet workload install wasm-tools from dotnet-wasm directory to …
Lxiamail Jul 16, 2021
b890b88
install wasm-tools workload from perf directory
Lxiamail Jul 16, 2021
29ab2eb
install wasm-tools workload with a specific nuget.config with wasm feed
Lxiamail Jul 16, 2021
ebbfe38
fix path
Lxiamail Jul 16, 2021
2f8b61e
workdir of install wasm-tools
Lxiamail Jul 20, 2021
fe45b23
Get PR#1891 from main
Lxiamail Jul 20, 2021
a35a0c7
update to new version of benchmarkdotnet for wasm aot fixes
Lxiamail Jul 21, 2021
8815bc5
add -d to print diagnostic info for wasm_tools workload installation …
Lxiamail Jul 22, 2021
a537566
Allow to build with local BenchmarkDotNet sources
radekdoulik Jul 23, 2021
1d11a28
Set _BenchmarkDotNetSourcesN conditionaly
radekdoulik Jul 23, 2021
ea28179
Use _BenchmarkDotNetSourcesN
radekdoulik Jul 23, 2021
4a4ca92
Use BenchmarkDotNet sources in payload
radekdoulik Jul 23, 2021
d566d6b
update to newer benchmarkdotnet version
Lxiamail Jul 26, 2021
a69d693
remove testing code
Lxiamail Jul 30, 2021
3ffe4d7
update bdn version
Lxiamail Jul 30, 2021
0132916
Merge branch 'main' into alicial/wasmaotmicro
Lxiamail Jul 30, 2021
7c5b5a1
remove duplicated section
Lxiamail Jul 30, 2021
badd062
print out args.configuration for diagnosis
Lxiamail Aug 2, 2021
6c383ce
print args
Lxiamail Aug 2, 2021
a6f66d3
print args
Lxiamail Aug 2, 2021
1025c15
minor fix
Lxiamail Aug 2, 2021
2056b08
looks up wasm in ExtraBenchmarkDotNetArguments
Lxiamail Aug 2, 2021
aee0c2e
check wasmEngine for wasm runs
Lxiamail Aug 3, 2021
c49c485
minor fix
Lxiamail Aug 3, 2021
272a41f
minor fix
Lxiamail Aug 3, 2021
4b2930b
remove work around code for install wasm_tools workload, WasmNativeWo…
Lxiamail Aug 3, 2021
72924ae
add back Directory.Build.props to see if can work around an aot build…
Lxiamail Aug 3, 2021
8b2dc7d
Merge branch 'main' into alicial/wasmaotmicro
Lxiamail Aug 4, 2021
56027a4
disable FlushAsync test from wasm run since async is not supported.
Lxiamail Aug 4, 2021
81bad40
Merge branch 'alicial/wasmaotmicro' of https://github.com/dotnet/perf…
Lxiamail Aug 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<Project>
<PropertyGroup>
<WasmNativeWorkload>false</WasmNativeWorkload>
</PropertyGroup>
</Project>
Comment on lines +1 to +6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this file in the docs folder?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. :) The goal is put this Directory.Build.props file to a root/parent directory where benchmarkdotnet runs from. Due to we copy docs directory to workitemdirectory, having Directory.Build.props file in the same directory makes it work. However, we should remove ASAP since this could have unexpected back consequences. opened #1909 to track this.

3 changes: 0 additions & 3 deletions scripts/benchmarks_ci.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,6 @@ def __is_valid_datetime(dt: str) -> str:
action='store_true',
help='Attempts to run the benchmarks without building.',
)

return parser


Expand All @@ -171,7 +170,6 @@ def __process_arguments(args: list):
add_arguments(parser)
return parser.parse_args(args)


def __main(args: list) -> int:
validate_supported_runtime()
args = __process_arguments(args)
Expand Down Expand Up @@ -270,6 +268,5 @@ def __main(args: list) -> int:
upload.upload(globpath, upload_container, UPLOAD_QUEUE, UPLOAD_TOKEN_VAR, UPLOAD_STORAGE_URI)
# TODO: Archive artifacts.


if __name__ == "__main__":
__main(sys.argv[1:])
4 changes: 1 addition & 3 deletions scripts/micro_benchmarks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,8 +313,7 @@ def run(
verbose,
*run_args
)



def __log_script_header(message: str):
getLogger().info('-' * len(message))
getLogger().info(message)
Expand Down Expand Up @@ -361,7 +360,6 @@ def __main(args: list) -> int:
verbose,
args
)

return 0
except CalledProcessError as ex:
getLogger().error(
Expand Down
6 changes: 6 additions & 0 deletions scripts/performance/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,12 @@ def helixpayload():
'''
return environ.get('HELIX_CORRELATION_PAYLOAD')

def helixuploadroot():
Lxiamail marked this conversation as resolved.
Show resolved Hide resolved
'''
Returns the helix upload root. Will be None outside of helix.
'''
return environ.get('HELIX_WORKITEM_UPLOAD_ROOT')

def runninginlab():
return environ.get('PERFLAB_INLAB') is not None

Expand Down
1 change: 1 addition & 0 deletions src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.File.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ public void SetupWriteAllBytes()
public void WriteAllBytes(int size) => File.WriteAllBytes(_testFilePath, _userBuffers[size]);

#if !NETFRAMEWORK
[BenchmarkCategory(Categories.NoWASM)]
[Benchmark]
[Arguments(HalfKibibyte)]
[Arguments(FourKibibytes)]
Expand Down
1 change: 1 addition & 0 deletions src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.FileStream.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ public void Flush(long fileSize, FileOptions options)
}
}

[BenchmarkCategory(Categories.NoWASM)]
[Benchmark]
[Arguments(OneKibibyte, FileOptions.None)]
[Arguments(OneKibibyte, FileOptions.Asynchronous)]
Expand Down
13 changes: 13 additions & 0 deletions src/benchmarks/micro/wasm/nuget.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This nuget,config is used when installs emsdk in performance-setup.sh file

<configuration>
<packageSources>
<clear />
<add key="dotnet-public" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json" />
<add key="dotnet6" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
<add key="xamarin" value="https://pkgs.dev.azure.com/azure-public/vside/_packaging/xamarin-impl/nuget/v3/index.json" />
<add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
<add key="dotnet-tools" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" />
<add key="benchmark-dotnet-prerelease" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/benchmark-dotnet-prerelease/nuget/v3/index.json" />
</packageSources>
</configuration>
2 changes: 1 addition & 1 deletion src/harness/BenchmarkDotNet.Extensions/BenchmarkDotNet.Extensions.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<PropertyGroup Condition="'$(BenchmarkDotNetSources)' != ''">
<_BenchmarkDotNetSourcesN>$([MSBuild]::NormalizeDirectory('$(BenchmarkDotNetSources)'))</_BenchmarkDotNetSourcesN>
</PropertyGroup>
<ItemGroup>
<ItemGroup Condition="'$(BenchmarkDotNetSources)' == ''">
<PackageReference Include="BenchmarkDotNet" Version="0.13.0.1579" />
<PackageReference Include="BenchmarkDotNet.Diagnostics.Windows" Version="0.13.0.1579" />
</ItemGroup>
Expand Down
2 changes: 1 addition & 1 deletion src/harness/BenchmarkDotNet.Extensions/RecommendedConfig.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ public static IConfig Create(
.DontEnforcePowerPlan(); // make sure BDN does not try to enforce High Performance power plan on Windows

// See https://github.com/dotnet/roslyn/issues/42393
job = job.WithArguments(new Argument[] { new MsBuildArgument("/p:DebugType=portable") });
job = job.WithArguments(new Argument[] { new MsBuildArgument("/p:DebugType=portable"), new MsBuildArgument("-bl:benchmarkdotnet.binlog") });
Lxiamail marked this conversation as resolved.
Show resolved Hide resolved
}

var config = ManualConfig.CreateEmpty()
Expand Down