Content-Length not set in ForwardResponseMessage

[Joshua Garnett]

Content-Length is not set before calling w.Write at https://github.com/grpc-ecosystem/grpc-gateway/blob/main/runtime/handler.go#L179. This ends up relying on the default Go behavior, which will only set Content-Length for small responses. When using CDNs like CloudFront they will not compress the origin responses unless Content-Length is set.

In theory the header could be added by a WithForwardResponseOption, the downside being that the message needs to be serialized twice once by the WithForwardResponseOption and once by the ForwardResponseMessage code. That said it could be incorrect if if rb, ok := resp.(responseBody); ok { a few lines up is true.

Is there a better way to do this? Thanks in advance!

[Johan Brandhorst-Satzkorn]

Thanks for your issue! I'd like to ask some clarifying questions:

1. Could you elaborate on the behavior of the net/http handler in this case? If they choose to only do this for "small responses", why should we do anything differently?
2. Is this something that could be by a custom middleware instead? I'm thinking something that wraps the http.ResponseWriter and counts the bytes written, setting the header before writing it to the wire. That would mean we wouldn't have to add it as an option here.
3. Thanks for the PR, but please refrain from submitting anything until we've discussed the best way forward.

[Joshua Garnett]

Hey @johanbrandhorst,

On question 1:

From https://github.com/golang/go/blob/960fa9bf66139e535d89934f56ae20a0e679e203/src/net/http/server.go#L125

	// If [ResponseWriter.WriteHeader] has not yet been called, Write calls
	// WriteHeader(http.StatusOK) before writing the data. If the Header
	// does not contain a Content-Type line, Write adds a Content-Type set
	// to the result of passing the initial 512 bytes of written data to
	// [DetectContentType]. Additionally, if the total size of all written
	// data is under a few KB and there are no Flush calls, the
	// Content-Length header is added automatically.

https://github.com/golang/go/blob/960fa9bf66139e535d89934f56ae20a0e679e203/src/net/http/server.go#L1582 also has some additional comments.

The library doesn't know if the calling code is going to call write once or multiple times. In this case, we know the Write method is only called once as long as doForwardTrailers isn't set.

On question 2:

I'm not sure of a way this can be done without writing the data to a buffer first, which adds additional allocations. This could be problematic for very large responses.

[Johan Brandhorst-Satzkorn]

Thanks for your responses. I wonder if we can't just enable this behavior by default since we know the size of the buffer we want to write. Worst case it's going to add a content-length header where we didn't before, but that seems like a better experience. What do you think? Feel free to update the PR and remove the option if you agree.

[Joshua Garnett]

I'm supportive of that, the only edge case I could think of is if a user already had some other middleware that was writing the header and maybe causing some trouble there. I'll update the PR now.

[Joshua Garnett]

I recreated the PR with a cleaner history