HTTP/2 support

[cactorium]

I'm guessing this would make good feature request :)

[pyfisch]

Is it possible with current blocking design to write a good HTTP/2 implementation?

[seanmonstar]

I don't see a specific stopper to implementing it with the blocking API.
However, we are exploring using async io for hyper before hyper 1.0.

On Sun, Feb 22, 2015, 11:01 AM Pyfisch notifications@github.com wrote:

Is it possible with current blocking design to write a good HTTP/2
implementation?

—
Reply to this email directly or view it on GitHub
#304 (comment).

[andydude]

I am interested in being assigned to this, but I might look around the hyper source first.

[DemiMarie]

Hyper now uses asynchronous I/O, so HTTP/2 is no longer blocked in async I/O.

[stepancheg]

I have kind of working HTTP/2 implementation: httpbis. ("httpbis" is a name of working group developing HTTP/2, "bis" traslates as "twice".)

https is used in my implementation of gRPC (which works over HTTP/2).

It is a fork of @mlalic's solicit library, which was largely rewritten. Now it contains missing HTTP/2 parts, and it is rewritten using tokio instead of being synchronous.

It implements most of HTTP/2 spec, including multiplexing and proper windowing.

Proper windowing means that client or server won't send WINDOW_UPDATE request untill Stream passed to user callback is polled. So for example, connection won't stall, and memory won't be exhaused if user provided callback is slow.

Current status is that it works, however, it needs lots of polishing, including:

• documenting
• testing
• finding and fixing bugs
• adding configuration options
• designinig better API
• improving performance

(And I could use help for these issues).

If someone's interested I'd like to merge it into Hyper somehow.

[seanmonstar]

@stepancheg wow, exciting! I admit to not having much time to review all of it.

I encourage you to keep working on it, including the list you provided. hyper is far too close to a 0.11 release to try to squeeze in such a huge change. But we can definitely see how the project goes!

[lilith]

HTTP/2 support is huge for performance. Globally 82% of clients support it - 93% in the USA.

Is HTTP/2 support planned for hyper 1.0?

[seanmonstar]

Yes, we're working on the h2 crate at the moment, and hyper 0.12 will be released with the h2 built in.

[avkonst]

Could you please advise when hyper 0.12 is coming?

[seanmonstar]

We wish to group together the breaking changes, and with tokio going through a reform, that is the current blocker.

[szagi3891]

What changes are planned in the "Tokyo"?

[rpjohnst]

@szagi3891 tokio-rs/tokio-rfcs#3

[avkonst]

That is unfortunate dependency :( Is it feasible to have 2 releases with the first one releasing this feature?

[carllerche]

@avkonst you can use the h2 crate as is today.

[avkonst]

Thanks. It seems it would require to go one level lower, than the level of abstraction provided by hyper + iron for me. I would expect that the web framework like Iron will benefit automatically from Hyper being http2 capable. Plus, it seems h2 does not handle http 1.1 and alpn, it says it is only http2 "parser".

[carllerche]

@avkonst I would say that calling h2 a "parser" is a bit unfair. We are using already with great success. As you can see here setting up ALPN is not hard. Also, you don't need http 1.1 to do prior knowledge.

That said, all the work is happening on a volunteer basis, so if the release doesn't match your timeline, I would suggest trying to pitch in ;)

[avkonst]

@carllerche I understand this is on volunteer basis 👍 I see big potential in Rust and trying to setup development with Rust, where I work. It turned out that the main objection I receive is not the learning curve, but availability of solid http stack. I understand that bits and pieces are there, which can be glued together into complete solution.

"solid" I (and people around me) mean, like Vert.x or Jetty are solid in Java: setup of a client or a server in few lines with reasonable defaults and completes (such as HTTP 2, streaming, pipes, async). In case of Scala layers over, all the APIs are async Future based too.

I understand that Rust will get the same API completeness one day, I see it happening now. However, making any business case today is hard without knowing the release date of essentials like this ticket. In my case, it would be easier to influence other people saying something like: "you know, I understand lack of http 2 support is a concern for us, but while we are developing new product with what is available today and trying it out with few customers, it would be perfectly fine. And in X months it will get upgraded to http 2 with minimal impact anyway".
If I knew this number X, it would be extremely helpful.

PS: one of the reasons of rapid Go adoption (although I do not like Go at all), was "solid" HTTP and networking stack.

[seanmonstar]

We are using the h2 crate in an internal project, allowing us to flesh it out as the project needs more HTTP2 features, and take care of bugs we find. Once the needs of the internal project are more satisfied, we will have time to polish and publish h2, and then get hyper using it internally.

No promises, but it's possible this could appear in hyper in about a month or two.

one of the reasons of rapid Go adoption (although I do not like Go at all), was "solid" HTTP and networking stack.

We're working on it! :D

[avkonst]

@seanmonstar Thank you! It is great to know this :)

[DemiMarie]

May I ask what the advantage of the h2 crate is over the rust-http2 crate?

[stepancheg]

@DemiMarie author of rust-http2 here. rust-http2 is buggy, poorly documented and barely maintained. I'd like to build a proper library from rust-http2, but I don't have time. rust-http2 was created as a base for grpc-rust. And I'm probably going to replace rust-http2 with h2 eventually in grpc-rust.

[seanmonstar]

An update: we've release h2 v0.1!

Now then, HTTP/2 support in hyper:

• hyper v0.11 has a minimum Rust dependency of 1.18.
• The h2 crate depends on the http crate, which requires associated constants, needing Rust 1.20.
• I've tried very hard to keep it so that patch releases of hyper don't break compilation for a user, and so don't require new compilers.
• So, by trying to do that, it seems likely that using the h2 crate will require a breaking change to hyper 0.12 (and can happen at the same time as Replace types with those from the http crate #1317).
• However, there are some upcoming breaking changes due to the tokio-reform, and it seems like it could be a good idea to group those 2 breaking releases into 1, just 0.12.

With just that, it'd seem like HTTP/2 support is still a little ways out. However, I think there's something we can do to get HTTP/2 into hyper v0.11:

• There is already the compat cargo feature which adds conversions to hyper types from types from the http crate.
• We can add internal usage of the h2 crate, by putting it behind an http2 cargo feature as well.
• This means current users of hyper v0.11 can freely cargo update and it still compiles.
• And users who want HTTP/2 can opt-in, knowing they need a newer compiler version, but setting hyper = { version = "0.11", features = ["http2"] }.
• Besides opting in, there wouldn't be a need to do anything else: the client and server would both be able to speak HTTP/2 automatically.
• It does mean that users of hyper v0.11 don't get HTTP/2 support automatically, which is a bummer, but 0.12 isn't that far away.

Does this seem like the right plan?

[mgattozzi]

This seems like a good plan. Allows opt in breakage and can let people experiment before the 0.12 release and can let them use http2 while those who are okay with waiting can continue with http1.1

[sfackler]

Seems reasonable to me. I think it'd take manual intervention for people to be able to take advantage of HTTP/2 anyway, since you'd need to poke at ALPN in the handshake and then tell Hyper which protocol to use.

[seanmonstar]

There is a working proof-of-concept in #1432. The small test suite passes, so if anyone wants to try it out, have fun 🐉. It only works with HTTP2 Prior Knowledge so far.

It might be that by the time its confident enough to merge, the breaking changes in tokio and futures will have released and we'll be making hyper 0.12, in which case this wouldn't even need a feature flag...

[seanmonstar]

#1432 was merged into master, providing HTTP/2 support for both the server and client. At first, it requires opting in, and only supports HTTP2 Prior Knowledge.

I'm going to open other issues that are more specific around more support, such as with ALPN, HTTP1 upgrades, and defaulting on.

🎉 🎊 🎈 ✨