Endpoints are missing a content-type header

[wperron]

Some of the API endpoints are returning a content-type of text/plain even if the Accept header on the request is set to application/json or application/protobuf. I noticed it on /api/traces/{traceID} and /api/search at least, there could be others as well.

[kvrhdn]

Thanks for creating this! The way Tempo deals with content encoding is a bit funky because we convert everything to protobuf internally. So the response coming from the querier will be in protobuf and the query-frontend is responsible for converting it into JSON if the user requested this.
It doesn't surprise me we forgot to set some headers along the way 😅

Some pointers about the code:

Trace lookup

This is where we determine the response content encoding:
https://github.com/grafana/tempo/blob/main/modules/frontend/frontend.go#L109-L113

And this is where we set the response body, we should probably set the content-type here as well:
https://github.com/grafana/tempo/blob/main/modules/frontend/frontend.go#L137-L151

Search

The search endpoint is a bit simpler because we only return search results in JSON.

We probably should set the content-type somewhere here: https://github.com/grafana/tempo/blob/main/modules/frontend/frontend.go#L163-L184

Which makes me think... the querier and ingester have the same/similar endpoints so we should also set this header there.

[wperron]

Which makes me think... the querier and ingester have the same/similar endpoints so we should also set this header there.

Probably won't hurt for sure, but as you said, everything is done over protobuf internally, so it would still require setting the content-type in the query frontend based on the accept header yeah?

[kvrhdn]

Yeah. that's true. I was thinking of a scenario in which the user requests protobuf, in that case we don't have to do any conversion in the query-frontend. But we actually still have to manipulate the body a little bit to return the correct format so we never forward the request as-is.