Requests greater than MTU fail to parse
mcholste opened this issue · comments
I know that the proper fix is to implement stream-based packet capture, but a simple truncation method for requests greater than the packet MTU would be much better than simply ignoring the request. Can this be added?
Do you have an example capture file that demonstrates this behavior? It shouldn't simply ignore any packet that's larger than the MTU, but it depends on how the data falls within the packet as to whether or not it's recognized as HTTP traffic. It would be helpful to see exactly what you're seeing on the wire to make sure we're talking about the same thing.
Thanks!
I can provide you an actual pcap privately if you give me contact info. The gist looks like this from an SQL injection:
0x0030: 014e fd8c 4745 5420 2fxxxxxxxxxxxxxxxx .N..GET./<redacted>
...
0x0060: 7369 643d 3135 3029 2b64 6563 6c61 7265 sid=150)+declare
0x0070: 2b25 3430 732b 7661 7263 6861 7225 3238 +%40s+varchar%28
0x0080: 3830 3030 2532 392b 7365 742b 2534 3073 8000%29+set+%40s
0x0090: 2533 4463 6173 7425 3238 3078 3733 3635 %3Dcast%280x7365
0x00a0: 3734 3230 3631 3665 3733 3639 3566 3737 7420616e73695f77
...
0x0570: 3330 3330 3239 3263 3562 3237 3262 3430 3030292c5b272b40
0x0580: 3433 3262 3237 3564 3239 3239 432b275d2929
18:44:10.207407 IP 64.191.99.83.41661 > .80: Flags [P.], seq 1369:1773, ack 1, win 46, options [nop,nop,TS val 1912592756 ecr 21953932], length 404
...
0x0030: 014e fd8c 3263 3331 3337 3239 3363 3365 .N..2c3137293c3e
0x0040: 3237 3237 3232 3365 3363 3266 3734 3639 2727223e3c2f7469
0x0050: 3734 3663 3635 3365 3363 3733 3633 3732 746c653e3c736372
0x0060: 3639 3730 3734 3237 3237 3230 3237 3239 6970742727202729
0x0070: 3230 3436 3435 3534 3433 3438 3230 3465 204645544348204e
0x0080: 3435 3538 3534 3230 3436 3532 3466 3464 4558542046524f4d
0x0090: 3230 3534 3631 3632 3663 3635 3566 3433 205461626c655f43
0x00a0: 3735 3732 3733 3666 3732 3230 3439 3465 7572736f7220494e
0x00b0: 3534 3466 3230 3430 3534 3263 3430 3433 544f2040542c4043
0x00c0: 3230 3435 3465 3434 3230 3433 3463 3466 20454e4420434c4f
0x00d0: 3533 3435 3230 3534 3631 3632 3663 3635 5345205461626c65
0x00e0: 3566 3433 3735 3732 3733 3666 3732 3230 5f437572736f7220
0x00f0: 3434 3435 3431 3463 3463 3466 3433 3431 4445414c4c4f4341
0x0100: 3534 3435 3230 3534 3631 3632 3663 3635 5445205461626c65
0x0110: 3566 3433 3735 3732 3733 3666 3732 2b61 5f437572736f72+a
0x0120: 732b 7661 7263 6861 7225 3238 3830 3030 s+varchar%288000
0x0130: 2532 3925 3239 2b65 7865 6325 3238 2534 %29%29+exec%28%4
0x0140: 3073 2532 392d 2d26 7762 7965 6172 3d32 0s%29--&wbyear=2
0x0150: 3030 3320 4854 5450 2f31 2e31 0d0a 5573 003.HTTP/1.1..Us
0x0160: 6572 2d41 6765 6e74 3a20 4d6f 7a69 6c6c er-Agent:.Mozill
0x0170: 612f 342e 3020 2863 6f6d 7061 7469 626c a/4.0.(compatibl
0x0180: 653b 204d 5349 4520 372e 303b 2057 696e e;.MSIE.7.0;.Win
0x0190: 646f 7773 204e 5420 362e 3029 0d0a 486f dows.NT.6.0)..Ho
0x01a0: 7374 3a20 xxxxxxxxxxxxxxxxxxxxxxxxxx st:.
0x01b0: xxxxxxxxxxxxxx0d 0a41 6363 6570 743a ..Accept:
0x01c0: 202a 2f2a 0d0a 0d0a ./....
Actually, what you included confirms what I suspected to be the issue. The problem is that httpry parses the packet line by line looking for HTTP data. Because the request URI is so long, it spans multiple packets, so httpry doesn't see even a single full line of HTTP data. I'll have to consider how to address this case in a way that also doesn't introduce false positives.
Thanks for pointing it out!
Yep, that's what I had surmised. I tried writing a patch, but I couldn't get anything to work. The behavior I expected would be to assume that all data up to the end of the truncated packet is URI and to send an incomplete request instead of no request.
Take a look at 92670ed and see if it resolves this issue for you. It essentially does exactly what you suggested: if no end-of-line character is present in the first line of the packet, it attempts to parse it anyway and will display whatever data is present.
I haven't had time to do extensive testing on it, but it appears to work as expected with no side-effects.
Seems to work perfectly! I've verified the proper truncation and haven't experienced any crashes under load. I consider the issue closed, thanks! FYI: I use httpry in my ELSA project: http://code.google.com/p/enterprise-log-search-and-archive/source/browse/trunk/elsa/contrib/httpry_logger.pl