[processor/transform] Add ability to limit attributes #9552
Conversation
|
This should be merged after #9546 |
|
@open-telemetry/collector-contrib-approvers @anuraaga this is now ready to be reviewed. |
There was a problem hiding this comment.
This can be a separate issue, but I wonder if the fully random behavior will limit the usefulness of this function. Let's say I have a small number of very important attributes but would need to limit the more arbitrary ones. It seems I have no control to preserve those important keys.
Perhaps something like limit(target, limit, priority_keys...) would be a natural extension for this case.
|
I like that idea. Any idea how instrumentations are dealing with the randomness of the drops? |
Looking through some instrumentations, it looks like the limits are applied as attributes are being added. If you add in a big bunch, Java will randomly drop, but if you add one at a time, the previously added attributes take precedence. For the collector, since most attributes are already applied, I think the addition of a "never drop these attributes" feature is a good add. I will open a new issue for this. |
Issue opened: #9734 |
|
@open-telemetry/collector-contrib-approvers please review. |
…#9552) * Add truncation function * Add TODO * Updated changelog * Updated changelog * Update name to truncateAll * Fixed README * add limit function * Updated changelog * Updated go.sum * Add check to avoid loop * added checked for negative limit * fix README.md * Fix lint issue * Add link to issue for writing logs Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
Description:
This PR adds to the transform processor the ability to limit the number of items in a map. The intended goal being the ability to limit the number of attributes and resource.attributes.
Link to tracking Issue:
#9366
This is one of two PR I have submitted for this issue.
Testing:
Added unit tests and tested locally running the collector and tracegen
Documentation:
Updated README