Streamified response with empty body ignores other response attributes
mdenkinger opened this issue · comments
Describe the bug
When a Lambda function in invoke mode RESPONSE_STREAM
and streamifyResponse=true
returns a response with an empty body (null
or ''
), then the other response attributes like statusCode or headers are ignored.
To Reproduce
Sample code:
import { createRequire } from 'module';const require = createRequire(import.meta.url);
import middy from "@middy/core";
import { Readable } from "stream";
function healthcheck() {
return () => {
return handler();
};
}
async function handler() {
return {
statusCode: 301,
body: '',
headers: {
location: 'https://example.com/'
}
};
}
var middyHandler = middy({
streamifyResponse: true
})
.handler(healthcheck());
export {
middyHandler as handler
};
Empty body cause the Lambda to respond with 200 instead if 301
{
statusCode: 301,
body: '',
headers: {
location: 'https://example.com/'
}
}
Response headers from client:
>
< HTTP/1.1 200 OK
< Date: Thu, 14 Mar 2024 17:23:24 GMT
< Content-Type: application/octet-stream
< Transfer-Encoding: chunked
< Connection: keep-alive
< x-amzn-RequestId: xxxxx
< X-Amzn-Trace-Id: root=xxx
<
e.g. blank body (one space) causes the Lambda to respond correct with 301 and also the headers are present:
{
statusCode: 301,
body: ' ',
headers: {
location: 'https://example.com/'
}
}
Response headers from
>
< HTTP/1.1 301 Moved Permanently
< Date: Thu, 14 Mar 2024 17:23:42 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: keep-alive
< x-amzn-RequestId: xxxx
< location: https://example.com/
< X-Amzn-Trace-Id: root=xxxx
<
Expected behaviour
The other response attributes like statusCode, headers,cookies are return with an empty body.
Environment (please complete the following information):
- Node.js: 20
- Middy: 5.2.0
- AWS SDK 3.533.0
Additional context
Directly to Function URL or through API Gateway?
Empty body cause the Lambda to respond with 200 instead if 301
This is an AWS issue, Middy doesn't override statusCodes within core
. Please open a support ticket with AWS, and point them here.
If AWS signals that they won't fix, we could change https://github.com/middyjs/middy/blob/main/packages/core/index.js#L58 to default to
as a workaround. I'll have to think on If it should always or only when statusCode is 204, 300 - 399, or others.
PS I recall 3xx statusCodes used to return 500s when it was first released. So, they may be willing to fix.
@willfarrell Thanks for checking. It’s through a Function URL. I’ll raise a case a AWS and then update this issue with the response.
While working on the AWS case, I dug into the middy code more closely. I think I found a potential problem in how middy handles strings with no content. Basically, when there's no content in the iterator generator does not yield and then the pipeline then does not process the HttpResponseStream. As far as I can see.
To fix this, we can make a simple change. By ensuring the generator always yields content (even if it's just once with no content), we can resolve the issue.
use while (position <= length)
instead of while (position < length)
.
Here's a sample code without middy, but with the algorithm to generate a string response in a stream-like manner.
import util from 'util';
import stream from 'stream';
const {Readable} = stream;
const pipeline = util.promisify(stream.pipeline);
export const handler = awslambda.streamifyResponse(async (event, responseStream, _context) => {
responseStream = awslambda.HttpResponseStream.from(responseStream, {
statusCode: 301, headers: {
location: 'https://example.com'
}
});
let handlerBody = null;
handlerBody = handlerBody ?? ''
// Source @datastream/core (MIT)
let handlerStream
if (handlerBody._readableState) {
handlerStream = handlerBody
console.log('its a stream')
} else if (typeof handlerBody === 'string') {
function* iterator(input) {
const size = 16384 // 16 * 1024 // Node.js default
let position = 0
const length = input.length
while (position <= length) {
yield input.substring(position, position + size)
position += size
}
}
console.log('its a string')
handlerStream = Readable.from(iterator(handlerBody))
}
if (!handlerStream) {
throw new Error('handler response not a ReadableStream')
}
await pipeline(handlerStream, responseStream);
});
I had some trouble setting up ava tests on my computer, so I couldn't create a pull request yet. 😞