VM: Better strategy for TryReserveInitialMemory on arm64 (jump stubs)

[EgorBo]

In #63842 we agreed that 128mb size is too small for the initial memory and might not fit typical real world apps. In this attempt I managed to get a good rate of successful attempts to reserve 500-700mb close to coreclr locally (~50% of test runs).

Also, I introduced a Release-internal flag to disable that behavior in order to get more stable results in microbenchmarks.

@jkotas @janvorli

[jkotas]

It does not sound right for the benchmarks to measure something else than what customers see.

[EgorBo]

The problem that currently on ARM we have quite shaky results and it's difficult to measure improvements from various changes, e.g. note this "ampere" line compared to x64 windows and Linux

I'll experiment on crank what my flags do to a series of measurements and how big RPS is when we're lucky to reserve a piece next to coreclr (and how often) - according to my previous measurements it's 5% larger in that case

[EgorBo]

and in fact, in like 50% cases apps won't be able to reserve such a big chunk next to coreclr so that flag simulates, probably, the most common case.

[EgorBo]

So here is what happens on our ampere machine (linux-arm64) we use for TE benchmarks:

app is started, we know the module base address:

coreclrLoadAddress = (UINT_PTR)PAL_GetSymbolModuleBase((void*)VirtualAlloc);

then we append "guessed" CoreClrLibrarySize size to it (100mb) assuming we try to allocate above the location of libcoreclr:

preferredStartAddress = coreclrLoadAddress + CoreClrLibrarySize;

then in a loop we try to reserve 2Gb of memory from preferredStartAddress by appending 128Mb to it every iteration (and decreasing "desired" allocation size by the same number):

do
{
    m_startAddress = ReserveVirtualMemory(pthrCurrent, (void*)preferredStartAddress, sizeOfAllocation);
    if (m_startAddress != nullptr)
    {
        break;
    }

    // Try to allocate a smaller region (and futher)
    sizeOfAllocation -= 128 * 1024 * 1024;
    preferredStartAddress += 128 * 1024 * 1024;
} while (sizeOfAllocation >= MemoryProbingIncrement);

// Fallback:
if (m_startAddress == nullptr)
{
    // We were not able to reserve any memory near libcoreclr. Try to reserve approximately 2 GB of address space somewhere
    // anyway...

it means that after just one of such iteration we already out of RELOC's reach, futhermore 100Mb for "guessed" coreclr size is too much, 16Mb works just fine for me locally.

Next problem that it seems to be very difficult to find spare 2Gb of memory nearby - I was never able to do so on my Apple M1 but it seems to be possible on our Ampere linux-arm64 but only after ~3 iterations of appending 128mb to the base address. For my Apple M1 I was able with some low probability to reserve 1GB nearby and some decent probability for 500-700Mb

[EgorBo]

using System;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

public class Program
{
    public static void Main(string[] args) =>
        BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);

    [Benchmark]
    [Arguments(0.4f)]
    public float MathTest(float x) => 
        MathF.Cos(x) + MathF.Sin(x) + MathF.Tan(x) + MathF.Sin(x);
}

`

Method	Job	Toolchain	x	Mean
MathTest	Job-FIVHQM	/Core_Root/corerun	0.4	23.797 ns
MathTest	Job-VDOWXN	/Core_Root_PR/corerun	0.4	10.667 ns

in 70% cases the results are like this, in the rest 30% results are the same (24ns) but the baseline never managed to reserve 2Gb close to coreclr.

[janvorli]

Why does the increment and decrement differ for arm?

[EgorBo]

Unified (and increased the initial size to 1Gb) - works pretty good on my Apple M1 and even better on linux-ampere (more hits of successful reservations)

[EgorBo]

in the worst case it does 8 probes (decreasing 1Gb by 128Mb each iteration and increasing startAddress by 8mb) and bails out

[EgorBo]

@janvorli @jkotas so does it look good? It now tries hard allocating 1Gb heap nearby and, according to logs, it manages to do so on our TE ampere linux machine with some good probability. It also does probing by increasing startAddress by 8mb each iteration and decreasing desired memory size by 128Mb (up to 8 iteration). On My M1 it usually manages to allocate around 500-700Mb heap and keeps relocs working.

I can remove the env.var I added if you don't think it brings value - I just wanted to experiment with it on dotnet/performance to reduce possible noise.

[janvorli]

I'd suggest removing the env var, even though the results of benchmarks are going to be more noisy, it will allow us to see what users really get.

[EgorBo]

I'd suggest removing the env var, even though the results of benchmarks are going to be more noisy, it will allow us to see what users really get.

Removed

[janvorli]

This ifdef is incorrect, we don't define TARGET_XARCH out of JIT. Can you please use
#if defined(TARGET_ARM) || defined(TARGET_ARM64) here?

[EgorBo]

Ah, yes, again I am using JIT's flags in the VM 😞 the incorrect define slightly regressed x64/x86 by decreasing preferable size from 2Gb to 1Gb. Thanks, fixed

[EgorBo]

@janvorli does it look good now?

[janvorli]

LGTM, thank you!