Improved Gateway HTTP request metrics

[gmasgras]

Wishlist

from @gmasgras

• Merging feat: register first block metric by default #8332 means that we can now track time to first block 🎉 however only the sum and count metrics are exported. It would be very useful to also export this metric as a histogram (eg unixfs_get_latency_seconds_bucket) so that we can calculate percentiles (p90, p95) and get a better understanding of outliers. A good set of buckets could be 100ms, 500ms, 1s, 2s, 3s, 5s, 8s, 13s

from @lidel

• Merging Gateway support for /ipfs/{cid}?format=car|raw|... #8234 will introduce response types other than Unixfs (block, car), and we are also planning native support for dag-json and dag-cbor on Gateways
  • This means we need solid metrics that are format agnostic (e.g. time to first block)
    • gw_first_root_block_get_latency_seconds with time to return the root block – /ipfs/{cid}, /ipns/{example.com} low utility, we care about first content block
    • gw_first_content_block_get_latency_seconds with time to first content block of specific resource – /ipfs/{cid}/some/file.jpg
    • histogram
  • nice to have: metrics per response type
    • unixfs file
    • generated directory listing
    • raw block
    • car stream
    • (futrure) dag-json, dag-cbor
    • request count and time to generate full response may be useful thing to track per type

[thibmeu]

Regarding the set of buckets, what is the reasoning behind these values?
I feel like the granularity in the seconds unit is important, while there is very little information for blocks that are retrieved quickly.

I would suggest 50ms, 100ms, 250ms, 500ms, 1s, 2s, 5s, 10s. This is still arbitrary, but I feel it could capture more information.

[gmasgras]

My reasoning for most of the values is that it's common to see buckets using the Fibonacci sequence. 30s and 60s are personal preference + they align with some of our timeouts on the gateways.