Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support emulated NUMA for BCM2711 and BCM2712 #6273

Draft
wants to merge 7 commits into
base: rpi-6.6.y
Choose a base branch
from

Conversation

popcornmix
Copy link
Collaborator

From: Tvrtko Ursulin tvrtko.ursulin@igalia.com

This series adds a very simple NUMA emulation implementation and enables
selecting it on arm64 platforms.

Obvious question is why? Short answer - it can bring a significant performance
uplift on Raspberry Pi 5.

Longer answer is that splitting the physical RAM into chunks, and utilising an
allocation policy such as interleaving, can enable the BCM2712 memory controller
to better utilise parallelism in physical memory chip organisation.

In more concrete numbers, testing with Geekbench 6 shows that splitting into four
emulated NUMA nodes can uplift the single core score of the benchmark by around
6%, and the multi-core by around 18%.

Code is quite simple and new functionality can be enabled using the new
NUMA_EMULATION Kconfig option and then at runtime using the existing (shared
with other platforms) numa=fake= kernel boot argument.

@popcornmix
Copy link
Collaborator Author

This PR also allows numa to be configured system wide (without directly using numactl) and enabled it by default through bootargs.

@pelwell I'm sure the bootargs change is better in a lower level file.

@pelwell
Copy link
Contributor

pelwell commented Jul 19, 2024

I'm sure the bootargs change is better in a lower level file.

It isn't possible to put some command line settings in a common file and add to/modify it elsewhere, but if there are some devices that share a common command line then we can put it in a common file and override it in its entirety where necessary.

As it happens, of the BCM2711 devices, only CM4S has a different command line, and BCM2712 devices share the same one.

Is this something you are thinking to enable by default on BCM2711 and BCM2712?

@popcornmix
Copy link
Collaborator Author

Is this something you are thinking to enable by default on BCM2711 and BCM2712?

Yes - enabled by default on BCM2711 and BCM2712 (ideally on all distributions) is the goal.

@pelwell
Copy link
Contributor

pelwell commented Jul 23, 2024

Bumped with a minor DT refactor.

mairacanal and others added 7 commits September 2, 2024 13:24
Add some common code for splitting the memory into N emulated NUMA memory
nodes.

Individual architecture can then enable selecting this option and use the
existing numa=fake=<N> kernel argument to enable it.

Memory is always split into equally sized chunks.

Signed-off-by: Maíra Canal <mcanal@igalia.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: “Rafael J. Wysocki" <rafael@kernel.org>
Allow selecting NUMA emulation on arm64.

Signed-off-by: Maíra Canal <mcanal@igalia.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: “Rafael J. Wysocki" <rafael@kernel.org>
Add numa_policy kernel argument to allow overriding the kernel's default
NUMA policy at boot time.

Syntax identical to what tmpfs accepts as it's mpol argument is accepted.

Some examples:

 numa_policy=interleave
 numa_policy=interleave=skip-interleave
 numa_policy=bind:0-3,5,7,9-15
 numa_policy=bind=static:1-2

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
system_heap.max_order=<uint>

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Signed-off-by: Dom Cobley <popcornmix@gmail.com>
Most 2711 devices and all 2712 device share common bootargs (command
lines). Make the common values shared defaults, overriding them were
necessary.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Signed-off-by: Dom Cobley <popcornmix@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants