Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nativeaot][tests] Add Native AOT iOS perf scenarios #85317

Merged

Conversation

kotlarmilos
Copy link
Member

@kotlarmilos kotlarmilos commented Apr 25, 2023

This PR updates the CI jobs to include the Native AOT iOS performance scenarios. The changes include building the NativeAOT iOS scenarios and running them, as well as updating the underlying scripts and project files to support it. Additionally, it separates the scenarios from the current Mono AOT sample measurements.

Additionally, it temporarily fixes regression to allow testing. The regression will be fixed on the main in a separate PR tracked in #80911.

Preliminary perf measurements for the HelloiOS app using Native AOT.

SOD - iOS HelloWorld Native AOT Zip Size

Metric Average Min Max
SOD - iOS HelloWorld Native AOT Zip Size 1727305.000 bytes 1727305.000 bytes 1727305.000 bytes

SOD - iOS HelloWorld Native AOT .app

Metric Average Min Max
SOD - iOS HelloWorld Native AOT .app 8075485.000 bytes 8075485.000 bytes 8075485.000 bytes
pub/nativeaot/HelloiOS.app/HelloiOS 8074496.000 bytes 8074496.000 bytes 8074496.000 bytes
pub/nativeaot/HelloiOS.app/Info.plist 981.000 bytes 981.000 bytes 981.000 bytes
pub/nativeaot/HelloiOS.app/PkgInfo 8.000 bytes 8.000 bytes 8.000 bytes

Device Startup - iOS HelloWorld Native AOT

Metric Average Min Max
Generic Startup 38.600 ms 34.000 ms 41.000 ms

@kotlarmilos kotlarmilos added this to the 8.0.0 milestone Apr 25, 2023
@kotlarmilos kotlarmilos self-assigned this Apr 25, 2023
@ghost
Copy link

ghost commented Apr 25, 2023

Tagging subscribers to 'os-ios': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

This PR updates the CI jobs to include the Native AOT iOS performance scenarios. The changes include building the NativeAOT iOS scenarios and running them, as well as updating the underlying scripts and project files to support it. Additionally, it separates the scenarios from the current Mono AOT sample measurements.

Additionally, it temporarily fixes regression to allow testing. The regression will be fixed on the main in a separate PR tracked in #80911.

Author: kotlarmilos
Assignees: kotlarmilos
Labels:

os-ios, area-NativeAOT-coreclr

Milestone: 8.0.0

@kotlarmilos
Copy link
Member Author

kotlarmilos commented May 30, 2023

This PR is ready for review. The performance results can be found in the dotnet-runtime-perf pipeline. The Mono performance runs have been expanded to include separate runs with and without symbols. Once this PR is merged, the following job names need to be updated in the dashboard.

Native AOT performance measurements:

  • SOD - iOS HelloWorld Native AOT .app Size symbols
  • SOD - iOS HelloWorld Native AOT .app Size nosymbols
  • SOD - iOS HelloWorld Native AOT Zip Size symbols
  • SOD - iOS HelloWorld Native AOT Zip Size nosymbols
  • SOD - Device Startup - iOS Native AOT HelloWorld symbols
  • SOD - Device Startup - iOS Native AOT HelloWorld nosymbols

Mono performance measurements:

  • SOD - iOS HelloWorld Mono .app Size llvm symbols
  • SOD - iOS HelloWorld Mono .app Size llvm nosymbols
  • SOD - iOS HelloWorld Mono .app Size nollvm symbols
  • SOD - iOS HelloWorld Mono .app Size nollvm nosymbols
  • SOD - iOS HelloWorld Mono Zip Size llvm symbols
  • SOD - iOS HelloWorld Mono Zip Size llvm nosymbols
  • SOD - iOS HelloWorld Mono Zip Size nollvm symbols
  • SOD - iOS HelloWorld Mono Zip Size nollvm nosymbols
  • Device Startup - iOS Mono HelloWorld llvm symbols
  • Device Startup - iOS Mono HelloWorld llvm nosymbols
  • Device Startup - iOS Mono HelloWorld nollvm symbols
  • Device Startup - iOS Mono HelloWorld nollvm nosymbols

@LoopedBard3 I will keep you posted.

@@ -29,8 +29,8 @@
<DevTeamProvisioning Condition="'$(DevTeamProvisioning)' == ''">adhoc</DevTeamProvisioning>
<EnableAppSandbox Condition="'$(EnableAppSandbox)' == ''">false</EnableAppSandbox>
</PropertyGroup>

<Import Project="$(CoreClrProjectRoot)nativeaot\BuildIntegration\Microsoft.NETCore.Native.targets" />
<!-- Fix temporary regression, will be fixed on the main in a separate PR: https://github.com/dotnet/runtime/issues/80911 -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@ivanpovazan ivanpovazan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Just a small note on "symbol-stripped" scenarios, that in those cases we are stripping local symbols only via strip -x

@kotlarmilos
Copy link
Member Author

LGTM! Just a small note on "symbol-stripped" scenarios, that in those cases we are stripping local symbols only via strip -x

Thanks! If we maintain consistency across all platforms and strip in the same way, it should be fine. If I recall correctly, we encountered an issue with global symbols stripping.

@kotlarmilos kotlarmilos merged commit 534d1df into dotnet:main Jun 1, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Jul 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants