Lighthouse in Chrome limits maximum parallel HTTP2 connections

[thasmo]

This baffled me for quite a while and I think I was able to pin it down to lighthouse being the problem. Also, I tested this multiple times and it seems to be reproducable.

Thanks for the awesome work on lighthouse!

Steps

1. Open URL which is served via HTTP2.
2. Open network panel in the devtools.
3. Disable cache and reload the page.
4. Open the audits panel and run any audit.
5. Run 1 to 3 again.

What is the current behavior?

• ad 3.) The network waterfall diagram shows quite all resources being downloaded in parallel. (HTTP2 multiplexing works)
• ad 4.) When inspecting the trace of the audit, maximum parallel connections seem limited to 6 or something. (HTTP2 multiplexing not working?)
• ad 5.) The network waterfall diagram shows parallel connections to all resources limited to 6 or something, even after the audit was finished/closed.

What is the expected behavior?

• ad 4.) I expect lighthouse to not limit parallel connections on HTTP2 for the audit.
• ad 5.) I expect Chrome to download all resources in parallel, but it seems the parallel connection restriction introduced by running a lighthouse audit is still in place.

Environment Information

Lighthouse: 4.2.0
Chrome: 74.0.3729.108 (Official Build) (64-bit) 
OS: Windows 10 Pro 1809 (64-bit)

[thasmo]

Afaik this also happens running lighthouse in Chrome on Ubuntu.

[patrickhulce]

Thanks for filing @thasmo! This is really interesting...

I can reproduce the behavior you're talking about (though oddly not 100% of the time, more like 80%). I can also intermittently reproduce this by recording a page load in the "Performance" tab of DevTools after having run Lighthouse (but not in the network panel and refreshing for some reason).

I mildly suspect that this might be a visualization/reporting only bug without affecting the actual timing of requests because the trace categories necessary to visualize the network requests in trace viewer are slightly different between LH and DevTools but that wouldn't fully explain what we're seeing so I'm not sure. Root cause will be tricky to nail down for a Chrome scheduling and/or trace visualization issue like this, but thanks very much for bringing to our attention!

As for what this means in practice...

If you are using Lighthouse with the default settings (simulated Slow 4G/Fast 3G throttling), then this will not end up significantly impact your performance numbers. The network requests are still handled using H2, and we disregard the amount of time spent queuing on our observed load when we simulate under the alternate conditions so the requests will not be subject to the same 6 parallel request limit you observe there.

If you're using Lighthouse with DevTools applied throttling, then I'm sorry to say it actually fails on H2 more severely than just the 6 parallel requests issue (it's request-based approach does not throttle subsequent requests).

If you're using Lighthouse with OS/packet-level throttling then you're probably being bitten by this in the metrics.

[juliquiron]

FYI PSI is using http/1.1 -> #7326

[patrickhulce]

Thanks @juliquiron but this issue is not related to PSI.

[paulirish]

I can't reproduce a difference in download behavior after running Lighthouse.

I tried something else out.. in a fresh chrome profile i used chrome://net-export/ and then loaded https://http2.akamai.com/demo (and sometimes just https://http2.akamai.com/demo/h2_demo_frame.html)

I see a few things. I see 6 sockets opened to h1 and just 2 opened to h2. (I haven't run Lighthouse at all, so this is conceivably default behavior).

I'm also assuming you weren't using applied throttling, because it's just too disruptive for this sort of analysis.

Given that, though.. I imagine it was a weird situation of visualization.

Will close for now.

If anyone can provide a URL where I can test this, I'd happily reopen.