dumping stream for 413
Raynos opened this issue · comments
You dump the stream at the start if content-length > limit but don't dump the stream if the limit is hit in the data handler.
In both cases you return a 413.
Why do you dump the stream ? Shouldn't the application level error handler call req.close() ?
for HTTP requests, if you send a response, the request is dumped anyway, so i'm not even sure if this is necessary. it's more for non-http streams.
philosophically, raw-body "consumes" the stream, so you shouldn't have to worry about anything (like memory leaks) once the callback completes.
in the data handler, the stream is already flowing so the stream is already being dumped.
Then there is a bug with your early returns because you call .resume() without adding the .on("data") handler that does the limiting logic
the problem with consuming the stream is that goes against the point of limit() if someone sends me 500 concurrent requests of 2GB bodies then I want to use the limit option to close those bad streams early. The consuming the entire stream philosophy means that there are 500 pipes pumping data through my process for no reason. This might not be a big deal.
yeah, i guess we should defer it to the app. for HTTP requests it shouldn't matter as long as you send a response right away, but for regular streams, you have to req.destroy() it otherwise you might get leaks or what not. we can just add that in the docs - different streams have to be handled differently. HTTP is 99% of the use-case, anyways.
what's the point of counting the bytes when you don't care about the body, though? it already returns an error.
True. If you handle the error by closing the response then the request should close. Although I'm not too certain about that, can a http stream be halfOpen ? i.e. close the writable side by still read from the readable side?
Should we pause the stream if we have an error or should we keep it flowing ? That's kind of the question.
i asked isaacs a while ago: https://twitter.com/jongleberry/status/332641228283863040
so it'll be dumped, but i don't know if that means the client can/will continue writing.
i'm not sure about pausing either. if a response is sent i don't think it'll matter. we can ask isaacs or just leave an issue open.
I asked in #libuv on IRC. you should hang out in IRC on #stackvm aswell
3:16 PM I ran into stream parser question
3:17 PM Let's say I'm parsing a readable stream and I've encountered an invalid sub sequence. The readable stream hasn't errored but the parser has errored
3:17 PM Should I keep the readable stream flowing or should I pause() it ?
3:18 PM should I emit a parser error and let the user decide whether to flow the stream or pause it or destroy it ?
3:20 PM Raynos: if you're going to let the user decide then definitely pause the stream.
It makes sense that we need to pause() if we want to let the user make a decision.
yeah, sounds fine to me. in connect and your body module, do you think we need to dump the request, though?
@jonathanong In my body module I'm going to keep the req paused. my defaultErrorHandler in my router sends stuff down the response so the req is going to get cleaned up.
streams are returned paused now: a3c29ca
i didn't add tests for this, though. you think it's necessary?
Adding tests is up to you.
You might want to test that an infinite stream with a limit of 1kb is never pumped completely for all edge cases in some kind of acceptance test. That's really what's important.
The best way to test that is to write a test program that has to exit naturally. If your piping an infinite stream then node test.js will never exit.
haha i'm not sure what that means. you mind adding tests for that too?
yay! 1.1.0