logging of request headers in decision logs

[benhass]

What is the underlying problem you're trying to solve?

Decision logs could include header fields from the request. This would improve the ability to understand decisions and other issues in production, where debug logging is not appropriate. I looked at system.log.mask upsert functionality but did not have access to input.headers there. Possibly a change similar to #1456 could address that limitation.

Describe the ideal solution

Ideally, logging of request headers in decision logs would be configurable. The user could decide which headers to mask using the existing system.log.mask rule.

Describe a "Good Enough" solution

Enable access to input.headers when defining rules for system.log.mask and use the existing upsert functionality to modify the decision log body.

Additional Context

[srenatus]

Traditionally decision logs record the inputs and results of policy decisions. I'm not sure I understand how the incoming request headers, which aren't available in the policy evaluation, can impact the decision making. If they can't, they don't belong into the decision logs, I think.

This does not mean that there's no value in improving tracing, and especially giving your decision logs a reference to tracing spans, like done recently in #5230. Would that functionality help you in debugging production issues, too?

[benhass]

especially giving your decision logs a reference to tracing spans

Yup! This is exactly why I'm requesting it. It sounds like #5230 is specific to the envoy plugin, is that right? I can think of some other use-cases where we might want to log out headers that tells us about the client or other general request metadata, but I'm happy enough if it is focused on the tracing use cases.

[ashutosh-narkar]

We've updated OPA's decision log format to now include the trace and span identifier. So it's not specific to the envoy plugin.

[benhass]

I'm not sure I understand how the incoming request headers, which aren't available in the policy evaluation, can impact the decision making

@srenatus they are used in system.authz so they might have some impact on access to opa, but I get your point and I understand it may not be optimal to include headers for many users.

I am happy to know you've likely addressed my main use-case, so will look forward to testing out the next release. Thanks all!