Various parsing errors request handling changes - needs extended fuzzing tests
sigaloid opened this issue · comments
cargo install cargo-fuzz
cargo fuzz run libfuzzer
@JEBailey your edits fail after being fuzzed, if you run these commands you can test it yourself. For example a lot of assertions of the buffer size in http_parser.rs fail if the data is less than a few bytes long. For now I've moved it to request-refactor branch.
Invalid HTTP/1.1 requests should throw an error, not crash the thread, even though technically it's the same result. It's a lot of stepping through the errors so here's my workflow.
Three terminal tabs.
- One with
cargo fuzz run libfuzzer. - Another in a directory with a simple vial project:
vial::routes! {
GET "/" => |_| {"Greetings, creature."};
}
fn main() {
vial::run!();
}and a Cargo.toml set up to load the library from my local drive and not github (so your edits can actually be tested).
vial = { path = "/home/user/vial" }
- The last terminal window will run netcat from in the
vial/fuzz/artifacts/libfuzzer/directory
- Start fuzzing.
- When a result is found, take note of the filename. Try minimizing it (
cargo fuzz tmin libfuzzer fuzz/artifacts/libfuzzer/[FILE_NAME]) first, though this isn't strictly necessary, it'll make it easier to report the bug as it will more clearly show what the bug is. - Then, in the second window, start the webserver with
RUST_BACKTRACE=FULL cargo run. - Then, in the third window, run
cat [FILE_NAME_OF_ARTIFACT] | nc localhost 7667. In the second window with the webserver, a full stack trace should show you the exact line the error is on. (The fuzzer will print the name of the file) - Then I open my editor and fix the problem by making a small edit (usually just bounds checking or similar). Save the file, rerun the server in the second window, and rerun the netcat command in the third window. See if the thread crashes. If it still crashes, continue trying to fix the bug. If it doesn't crash, congrats! You've fixed one of the bugs.
- Rerun process by restarting the fuzzer in the first window. It should find another bug quickly. Sometimes you may need to leave the fuzzer running for a long time. (I had it running for 18 hours straight on a GCP VM without any problems on master before this PR, I'd say that long means you're probably fine)
- You may want to try giving it a sample directory instead of making the LLVM fuzzer have to relearn the codepaths again and again. You can do this by adding
-- [directory_of_vial]/tests/httpto the fuzzing command which will pass that command through to libFuzzer.
If you can work on this for a few minutes you probably will get to a point where the fuzzer could run for hours without hitting a crash. If you get the chance to do this, that would be appreciated so that I can remerge into master (and I'll probably set up an old Raspberry Pi to fuzz as well). If not, I'll try to get around to it this weekend, but as per status, I'm very crunched for time 😓 there's not really a rush for time though.
Cool. I haven't used the fuzzer before so this may take a bit to catch up with your explanation. I'll aim for the next day or two.
No rush! I can help if you get stuck at all, and there isn't a rush at all because as-is, it works alright, it just crashes the thread. Take your time!
Actually, if you haven't started on it - I'd like to post a video or maybe article on how to fuzz rust and it might be more beneficial to do it on a repo with real fuzzing errors. :D
That would be really cool. I'd appreciate seeing it.
So I was looking at the code and I'm sure I know what's causing the problem. I should have a line right before I parse for the method name that says peek!(7) this would prevent an out-of-range error from occurring when an incomplete method name is sent. I just need to implement and fuzz. I'm hoping to get some tonight to get this all validated.
That works! Feel free to commit it. If you'd like me to check and fuzz it, I can do that.
So I'm going to have to go back and change more things. My bad. I completely misunderstood how the WouldBlock error would surface in a non-blocking stream. I'm going to have to put back the Partial error message in the request parsing, so that I can re-attempt if I don't get enough data.
That works as well! No rush 😸
Most of the issues were out of bounds - fixed by matching with a buffer.get() in a lot of places. Running tests for a long time now, just to make sure it's still workign.
Merged :D closing
Cool. I tried doing the fuzzer but I'm getting an error when I attempt to run it. I'm thinking I need to run it in a particular directory?
If you have the fuzzer installed, you should be able to run cargo fuzz run libfuzzer in the vial/ directory and it should work.