Specify/override decompression algorithm
Sherlouk opened this issue · comments
Currently decompression relies on the upstream API you're downloading to set the 'Content-Encoding' header to 'gzip'. This doesn't always happen.
In fact, a request I'm making to an AWS S3 bucket to download a gzipped file only returns a Content-Type of application/x-gzip and so does not trigger this behaviour.
It would be nice if we had an ability to override this - still taking advantage of the decompression functionality.
This might be a feature as part of #393 (supporting middlewares)?
For reference, I have tried setting the header after the response has been returned (but before the body stream is processed) and it does not alter the decompression state.
For added context, by updating HTTPResponseDecompressor.swift I was able to force gzip decompression through async-http-client.
The solution here feels that we need the ability to manipulate the headers before it's sent to nio-extras for decompression (in essence, let me override the Content-Encoding header).
Yeah, middleware support would be valuable here. In the short term, we could update the HTTP compression code in NIOExtras to treat gzip and x-gzip as identical.
I'm still not sure that would work as the x-gzip code comes from ContentType not ContentEncoding.
nio-extras only looks at the latter which isn't set by S3 in my test examples.
Ah yes, good catch. We wouldn't normally want to transparently decompress a Content-Type: gzip, as it is likely that this file is intended to be downloaded as a gzip. Content-Encoding implies that the compression is not a fundamental part of the data.
Nonetheless, yes, I think this is something a middleware could solve.
On that note is it worth closing this down?
Its tagged with the middleware ticket in case we need to refer to it in the future, but doesn't appear to hold value on its own.
Sounds reasonable to me, thanks for filing this!