Ember server returns 500 on too many headers, doesn't say why

[Ross A. Baker]

1. Run an ember server with defaults
2. Send it more than 40k of headers
3. Response is a 500 with Content-Length: 0 and no mention of what's wrong in the logs

This is a bad request and should generate a 4xx error, perhaps a 431.

[Ross A. Baker]

I can't find a spec this violates, so "bug" is probably too harsh. But it's a real bad default.

Note that a server can handle this on 0.23.25:

          .withErrorHandler {
            case _: EmberException.MessageTooLong =>
              Response(Status.RequestHeaderFieldsTooLarge)
                .putHeaders(org.http4s.headers.`Content-Length`.zero)
                .pure[IO]
          }

An obvious solution is then to add this to the default error handler. Two concerns:

1. Because the default is so light, one wonders how many existing services configure their own without appending the default. It would be easy to obliterate.
2. Inconsistent with the solution to #6934, which special cases an error (and neatly provides a hook for an otherwise untyped, undocumented error).

On the other hand, this calls into question which other EmberExceptions merit a more thoughtful response than 500, and if we special case them each with an option, where does it end?

[Daniel Esik]

Could we suggest that this not only doesn't handle when the number of headers is that huge but also when a single header has an enormous length?

In any case, I concur that such default behaviour isn't fine. A 431 code as a response seems more reasonable.
Additionally, I'd say it's rather a bug (in terms of the spec), but who am I to judge?

[Ross A. Baker]

Actually, it is a bug per HTTP specification:

A server that receives a request header field line, field value, or set of fields larger than it wishes to process MUST respond with an appropriate 4xx (Client Error) status code.

I don't think Ember has a limit on a single field. If it did, it would also need to be a 4xx. I'm more concerned about the overall size, so I think the limit is good, and it's the behavior that's not.

I'm still afraid that the solution in #6934 doesn't scale to all the server responses we may wish to customize, but I could be persuaded otherwise.