[android][infra] Runtime tests on Android are not getting sent to Helix

[matouskozak]

Some recent changes likely from this range 516e81e...57a4b04 caused that Android runtime tests are not properly bundled and sent Helix (SendHelixJob given no WorkItems to send.)

Affected CI:

• android-x64 Release AllSubsets_Mono_RuntimeTests minijit
• android-x64 Release AllSubsets_Mono_RuntimeTests_Interp monointerpreter

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @directhex, @matouskozak
See info in area-owners.md if you want to be subscribed.

[matouskozak]

@jkoritzinsky do you think that #107200 could be the cause? I thought that the changes shouldn't affect Mono/Mobile but only this and #107167 from the range looks to be infra related.

[jkoritzinsky]

Unless Android was only "running" those tests (which are skipped on Android) and not running any other runtime tests, it couldn't be those. Can you check which tests were running beforehand?

[matouskozak]

Unless Android was only "running" those tests (which are skipped on Android) and not running any other runtime tests, it couldn't be those. Can you check which tests were running beforehand?

I think only the profiler tests where running:

[jkoritzinsky]

Well that's really not good....

I'll see if I can get these jobs back to actually doing work. If you want to get these pipelines green in the meantime, you can disable these jobs as you aren't actually losing any test coverage...

[matouskozak]

Well that's really not good....

I'll see if I can get these jobs back to actually doing work. If you want to get these pipelines green in the meantime, you can disable these jobs as you aren't actually losing any test coverage...

I agree and I'm unsure why we only used to run profiler tests even though we appear to be building the rest as well. I'll be OOF starting tomorrow but I'll check it out after I come back.

Thank you for looking into it.

[jkoritzinsky]

I found the issue. There was a change to fix an issue in running the runtime tests on desktop platforms about a year and a half ago which broke all mobile platforms. A later fix re-enabled WASM. I'm using that same fix (as it will work for our use case) for Android.