eBPF/PSA: Implement caching for ActionSelector, LPM and ternary tables #3738
Conversation
This commit introduces optimization for the PSA/eBPF backend called `table caching` which adds cache table with only `exact` match keys for time consuming look-ups. We identified three situations where this optimization can improve performance: - Table with `ternary` (and/or `lpm`, `exact`) key - skip slow TSS algorithm if the key was earlier matched. - Table with `lpm` (and/or `exact`) key - skip slow `LPM_TRIE` map (especially when there is many entries) if the key was earlier matched. - `ActionSelector` member selection from group - skip slow checksum calculation for `selector` key if it was earlier calculated. This optimization may not improve performance in every case, so it must be explicitly enabled by compiler option `--table-caching`. Limitations/to do: - `DirectCounter` and `DirectMeter` are not supported by this optimization (tables with these externs will not have enabled cache). - Cache size is a half of table size, but it would be more configurable or smart during compilation. - Updates to tables with cache or `ActionSelector` require cache invalidation (`nikss` library will remove all cached entries). Co-authored-by: Tomasz Osiński <osinstom@gmail.com>
| - table with `lpm` (and/or exact) key - skip slow `LPM_TRIE` map (especially when there is many entries) if the key was matched earlier. | ||
| - `ActionSelector` member selection from group - skip slow checksum calculation for `selector` key if it was calculated earlier. | ||
|
|
||
| The fast exact-match map is added in front of each instance of a table that contains a `lpm`, `ternary` or `selector` match |
There was a problem hiding this comment.
I also expect that modifying the original table may invalidate the cache.
There was a problem hiding this comment.
Cache invalidation is up to control plane. I think it never occur from eBPF program side.
| table entry. Tables with these externs will not have enabled cache optimization even when enabled by compiler option. | ||
| - When table cache optimization is enabled for a table, the number of cached entries is determined as a half of table size. | ||
| This would be more configurable or smart during compilation. | ||
| - Updates to tables or ActionSelector with enabled table cache optimization require cache invalidation. `nikss` library |
There was a problem hiding this comment.
You may be able to do better if you keep enough state to figure out what to invalidate.
Invalidating the whole cache may be very expensive.
There was a problem hiding this comment.
I agree, it might be expensive. I think that key and mask is enough to implement selective cache invalidation when updating table. Whole cache is invalidated because it is a simple algorithm and always work, but might be changed in the future.
| } | ||
|
|
||
| void EBPFTablePSA::emitCacheLookup(CodeBuilder* builder, cstring key, cstring value) { | ||
| cstring cacheVal = "cached_value"; |
There was a problem hiding this comment.
if this behaves like a table could you have reused the code generation for a table by creating a fake table?
There was a problem hiding this comment.
I can say cache behaves like a table but not in the same way. Both have similarly named methods, but internals of these are different. So table's method have to be overridden by these methods for cache (this code will not disappear). And the main part of table lookup is done by control block (not a table itself), so it is not easily accessible from table.
I think it will be rather hard to reuse code for table, so I would keep as is.
|
|
||
| builder->emitIndent(); | ||
| if (memcpy) { | ||
| builder->appendFormat("memcpy(&%s.%s, &", cacheKeyVar.c_str(), fieldName.c_str()); |
There was a problem hiding this comment.
so ebpf supports memcpy... or perhaps this is just an intrinsic that the compiler knows about
There was a problem hiding this comment.
Probably it maps to an intrinsic that the compiler knows about. I forgot to change it to __builtin_memcpy.
|
@mbudiu-vmw |
This PR introduces optimization for the PSA/eBPF backend called
table cachingwhich adds cache table with onlyexactmatch keys for time consuming look-ups. We identified three situations where this optimization can improve performance:ternary(and/orlpm,exact) key - skip slow TSS algorithm if the key was earlier matched.lpm(and/orexact) key - skip slowLPM_TRIEmap (especially when there is many entries) if the key was earlier matched.ActionSelectormember selection from group - skip slow checksum calculation forselectorkey if it was earlier calculated.This optimization may not improve performance in every case, so it must be explicitly enabled by compiler option
--table-caching.Limitations/to do:
DirectCounterandDirectMeterare not supported by this optimization (tables with these externs will not have enabled cache).ActionSelectorrequire cache invalidation (niksslibrary will remove all cached entries).