Proposal: change the default value of `initial_max_send_streams` to 100
magurotuna opened this issue · comments
Issue
Currently, the number of streams opened by the client before it receives the initial SETTINGS
frame from the server is set to usize::MAX
by default.
Line 645 in 122091a
This is valid according HTTP/2 spec (in fact the spec doesn't mention any explicit limit here). However, opening unlimited number of streams can lead to majority of them being rejected by the server with REFUSED_STREAM
, if the server's SETTINGS_MAX_CONCURRENT_STREAMS
is set to any lower value. This is simply not ideal and waste of resources.
Proposal
To better handle this issue, I propose that we start with reasonably low number of streams by default, rather than the maximum possible number. The initial number is only used until the client receives the initial SETTINGS
frame, so it should be much more preferable to start with a low number in order to reduce the risk of REFUSED_STREAM
errors even if it might end up with lower utilization for a very short time.
What is the reasonable default number? The good number should be as low as what most servers would accept. In fact, the spec recommends that the value should be no smaller than 100. To back up this recommendation, I've done a quick research on the values of MAX_CONCURRENT_STREAMS
used by several popular servers. The result is as follows.
Server | MAX_CONCURRENT_STREAMS |
---|---|
google.com | 100 |
youtube.com | 100 |
x.com | 100 |
facebook.com | 100 |
www.amazon.com | 100 |
www.apple.com | 100 |
netflix.com | 100 |
microsoft.com | 100 |
cloudflare.com | 100 |
fastly.com | 100 |
hyper.rs | 100 |
openai.com | 128 |
www.rust-lang.org | 128 |
crates.io | 128 |
vercel.com | 250 |
tokio.rs | 256 |
deno.com | (unspecified) |
So if the client starts with 100 streams at maximum, all of these servers will accept them and the client will not receive any REFUSED_STREAM
errors. Therefore we can conclude that 100 is the reasonable default number of streams to start with.
Take a quick look at other HTTP/2 implementations
To further support the proposal, let's have a look at how other HTTP/2 client implementations are doing. In short, all of these three implementations adopt starting with the reasonably small number of streams.
nghttp2
nghttp2's default initial maximum number of concurrent streams is 100.
https://github.com/nghttp2/nghttp2/blob/837f0c67c7139aedfb0c31d252634c823bfa75ba/lib/includes/nghttp2/nghttp2.h#L2537-L2554
Go x/net/http2
Go also uses 100 as the the initial number of maximum concurrent strams.
https://cs.opensource.google/go/x/net/+/689bbc7005f6bbf9fac1a8333bf03436fa4b4b2a:http2/transport.go;l=56
One notable difference from nghttp2 is that Go will set 1,000 as the MAX_CONCURRENT_STREAMS
if the server doesn't provide one, while nghttp2 sets inifinity in this case.
Python httpx
httpx starts with just 1.
https://github.com/encode/httpcore/blob/2fcd062df71555cc7de55774c6dc137551eb8692/httpcore/_async/http2.py#L117-L119
This is a great write-up, thank you! I agree it makes sense to adjust the default. I only want to pause to make sure it's the right place to adjust the default.
It's worth calling out that for workloads where the server will NOT place a limit, changing this default here could be a negative. This would be places where both peers are trusted. My mind goes to things like linkerd2 proxies. (cc @hawkw @olix0r) The proper solution could actually be that those use cases should explicitly set a high initial value, instead of depending on the default, which perhaps should aim to protect the most common cases.
But I also vaguely feel like so far we've put those defaults in hyper or reqwest, and left h2
to simply implement the protocol, and otherwise not pick defaults.
It makes sense that changing the default value could have a negative impact in a rare situation.
Actually, I opened an issue to here instead of to hyper (or reqwest) because I found that the client builder in h2 provides a way to set the initial MAX_CONCURRENT_STREAMS value while hyper doesn't. Plus, it looks like hyper doesn't set any value for initial MAX_CONCURRENT_STREAMS when constructing h2 client, thus ending up using the default value specified in h2, i.e. usize::MAX
.
So probably what we would like to do is:
- to have hyper provide a method on
hyper::client::conn::http2::Builder
that allows users to set the initial MAX_CONCURRENT_STREAMS value - to have hyper call
h2::client::Builder::initial_max_send_streams
when constructing h2 client with the user-configured value, if any.- If no value is configured by the user, set the reasonable default value i.e. 100.
- Or we could set the default value at one more upper level, i.e. reqwest.
- to have hyper_util provide a method on
hyper_util::client::legacy::Builder
that allows users to set the initial MAX_CONCURRENT_STREAMS value
I also feel that some changes still need to be made to h2, namely:
- improves the doc of
h2::client::Builder::initial_max_send_streams
to clearify that the default value isusize::MAX
- sets MAX_CONCURRENT_STREAMS to
usize::MAX
or whatever reasonable value (for example Go adopts 1,000) if the server's initial SETTINGS frame doesn't include MAX_CONCURRENT_STREAMS, assuming that the absence of MAX_CONCURRENT_STREAMS implies the server is willing to accept infinite number of concurrent streams- note that currently h2 keeps using the value set by
h2::client::Builder::initial_max_send_streams
if no specific value is supplied by the server
- note that currently h2 keeps using the value set by
Do these sound reasonable to you? If they do, I'd be happy to work on these :)
But I also vaguely feel like so far we've put those defaults in hyper or reqwest, and left h2 to simply implement the protocol, and otherwise not pick defaults.
+1. h2
definitely should concern the protocol-level implementation only. The defaults should be in hyper
.
Having arbitrary defaults everywhere is also a nightmare to handle, so it's good that we're avoiding that.
to have hyper call h2::client::Builder::initial_max_send_streams when constructing h2 client with the user-configured value, if any.
This sounds like it should be exposed in hyper
Thanks for your opinion @dswij.
This sounds like it should be exposed in hyper
What do you mean by this? If what you mean is that the users of hyper
should be able to configure the initial value of max send streams, then I think it's covered by the first bullet:
- to have hyper provide a method on hyper::client::conn::http2::Builder that allows users to set the initial MAX_CONCURRENT_STREAMS value
What do you mean by this? If what you mean is that the users of hyper should be able to configure the initial value of max send streams, then I think it's covered by the first bullet
Yes :)
I'm completely on board with the first point. Just saying that regardless of changing the default value, this option should be configurable in hyper
. So I see this as another improvement if we go down this route.
So I separated #732 into a couple of PRs. Now the part of setting a reasonable default value is moved to hyper. Here are the separated PRs:
Thanks a lot for doing that, your contributions have been wonderful to work with