Skip to content

Commit e3c72ec

Browse files
committed
Merge: CNB101: unroll: add generic loop unroll helpers
MR: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-10/-/merge_requests/921 JIRA: https://issues.redhat.com/browse/RHEL-92674 ``` commit c6594d6 Author: Alexander Lobakin <aleksander.lobakin@intel.com> Date: Thu Feb 6 19:26:26 2025 +0100 unroll: add generic loop unroll helpers There are cases when we need to explicitly unroll loops. For example, cache operations, filling DMA descriptors on very high speeds etc. Add compiler-specific attribute macros to give the compiler a hint that we'd like to unroll a loop. Example usage: #define UNROLL_BATCH 8 unrolled_count(UNROLL_BATCH) for (u32 i = 0; i < UNROLL_BATCH; i++) op(priv, i); Note that sometimes the compilers won't unroll loops if they think this would have worse optimization and perf than without unrolling, and that unroll attributes are available only starting GCC 8. For older compiler versions, no hints/attributes will be applied. For better unrolling/parallelization, don't have any variables that interfere between iterations except for the iterator itself. Co-developed-by: Jose E. Marchesi <jose.marchesi@oracle.com> # pragmas Signed-off-by: Jose E. Marchesi <jose.marchesi@oracle.com> Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com> Link: https://patch.msgid.link/20250206182630.3914318-2-aleksander.lobakin@intel.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> ``` Signed-off-by: CKI Backport Bot <cki-ci-bot+cki-gitlab-backport-bot@redhat.com> --- <small>Created 2025-05-20 12:18 UTC by backporter - [KWF FAQ](https://red.ht/kernel_workflow_doc) - [Slack #team-kernel-workflow](https://redhat-internal.slack.com/archives/C04LRUPMJQ5) - [Source](https://gitlab.com/cki-project/kernel-workflow/-/blob/main/webhook/utils/backporter.py) - [Documentation](https://gitlab.com/cki-project/kernel-workflow/-/blob/main/docs/README.backporter.md) - [Report an issue](https://issues.redhat.com/secure/CreateIssueDetails!init.jspa?pid=12334433&issuetype=1&priority=4&summary=backporter+webhook+issue&components=kernel-workflow+/+backporter)</small> Approved-by: Ivan Vecera <ivecera@redhat.com> Approved-by: Michal Schmidt <mschmidt@redhat.com> Merged-by: Julio Faracco <jfaracco@redhat.com>
2 parents 1d4a65a + 9ab36f6 commit e3c72ec

File tree

1 file changed

+44
-0
lines changed

1 file changed

+44
-0
lines changed

include/linux/unroll.h

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,50 @@
99

1010
#include <linux/args.h>
1111

12+
#ifdef CONFIG_CC_IS_CLANG
13+
#define __pick_unrolled(x, y) _Pragma(#x)
14+
#elif CONFIG_GCC_VERSION >= 80000
15+
#define __pick_unrolled(x, y) _Pragma(#y)
16+
#else
17+
#define __pick_unrolled(x, y) /* not supported */
18+
#endif
19+
20+
/**
21+
* unrolled - loop attributes to ask the compiler to unroll it
22+
*
23+
* Usage:
24+
*
25+
* #define BATCH 8
26+
*
27+
* unrolled_count(BATCH)
28+
* for (u32 i = 0; i < BATCH; i++)
29+
* // loop body without cross-iteration dependencies
30+
*
31+
* This is only a hint and the compiler is free to disable unrolling if it
32+
* thinks the count is suboptimal and may hurt performance and/or hugely
33+
* increase object code size.
34+
* Not having any cross-iteration dependencies (i.e. when iter x + 1 depends
35+
* on what iter x will do with variables) is not a strict requirement, but
36+
* provides best performance and object code size.
37+
* Available only on Clang and GCC 8.x onwards.
38+
*/
39+
40+
/* Ask the compiler to pick an optimal unroll count, Clang only */
41+
#define unrolled \
42+
__pick_unrolled(clang loop unroll(enable), /* nothing */)
43+
44+
/* Unroll each @n iterations of the loop */
45+
#define unrolled_count(n) \
46+
__pick_unrolled(clang loop unroll_count(n), GCC unroll n)
47+
48+
/* Unroll the whole loop */
49+
#define unrolled_full \
50+
__pick_unrolled(clang loop unroll(full), GCC unroll 65534)
51+
52+
/* Never unroll the loop */
53+
#define unrolled_none \
54+
__pick_unrolled(clang loop unroll(disable), GCC unroll 1)
55+
1256
#define UNROLL(N, MACRO, args...) CONCATENATE(__UNROLL_, N)(MACRO, args)
1357

1458
#define __UNROLL_0(MACRO, args...)

0 commit comments

Comments
 (0)