Use APM RUM agent to pass Kibana execution context

[mshustov]

In #101587, Kibana is going to use a custom kbn-context header to pass execution context during a request from the Kibana browser app to the Kibana server.
With elastic/apm-agent-rum-js#1040, we should be able to remove the custom header and switch to APM RUM agent API that will allow associating context metadata with an APM transaction. APM RUM agent will take care of serialization and passing context via proper tracing headers.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[felixbarny]

APM RUM agent will take care of serialization and passing context via proper tracing headers.

This will only work once our agents implement support for baggage. This requires a new API to add baggage, as well as header propagation.

[mshustov]

This will only work once our agents implement support for baggage.

@felixbarny Won't APM RUM agent propagate a bound context?
I agree that baggage header has the right semantic, we can wait for elastic/apm-agent-nodejs#2102 then. Btw, shouldn't the APM RUM agent open a similar issue to support the baggage header effort?

[felixbarny]

Won't APM RUM agent propagate a bound context?

No, our agents only propagate the traceparent header which contains correlation IDs. It looks like this:

traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01
(_________)  () (______________________________) (______________) ()
     v       v                 v                        v         v
  Name    Version           Trace-Id                Parent-Id     Flags

All the labels are not propagated to downstream services. Instead, each agent directly sends it's spans to APM Server. Using the trace-id it's then possible to correlate the different events.

[mshustov]

No, our agents only propagate the traceparent header which contains correlation IDs

ok, I wrongly assumed our agent propagates the standard tracestate header as well. Which can be used to pass custom data.

[felixbarny]

Yep, we transmit that, too. But that header is not meant for user-definable key/value pairs. We use that internally to propagate the sampling rate so that we can upscale metrics.

[lizozom]

@mshustov isn't this resolved by #124996 ?

[lizozom]

I think this is made obsolete by #102629 and other recent work.
Closing for now, please reopen if I got it wrong.