Improve dynamic analysis tooling

[Sergey "Shnatsel" Davidoff]

We already have some excellent tooling for generating test cases for programs to make them blow up:

• Parametric testing: QuickCheck and Proptest
• Fuzzing: AFL, libfuzzer and honggfuzz

We also want to add Angora to this collection which has raised the bar on fuzzing, see #17.

However, all of this glory is pretty much for naught if we don't have tooling to detect the runtime faults that these tests trigger. The current state is:

• Address Sanitizer is great, but requires workarounds to work (see #20)
• Memory Sanitizer is lacking documentation and is very hard to use with Rust (see #21)
• MIRI doesn't handle arbitrary Rust code, is not easy to use, does not have a clear value proposition compared to sanitizers and does not compose with fuzzers
• Nobody knows if Valgrind actually works now that Rust has ditched jemalloc, but it also doesn't compose with fuzzers, which limits its usefulness.
• Obscure tools such as DUMA are obscure for a reason.
• Bespoke tools such as libdiffuzz are not applicable in the general case, and are not easy/straightforward/foolproof enough for general use.
• Rust-specific tooling that works on actual compiled binaries pretty much does not exist.

Getting Address Sanitizer to work without workarounds (see #20) would already put us in an excellent position. Getting Memory Sanitizer to work would be harder, it is not universally applicable (see #21). Perhaps a more usable Rust-specific alternative could be concocted.

[Sergey "Shnatsel" Davidoff]

FWIW I've taken a stab at "universally applicable alternative to Memory Sanitizer" by writing libdiffuzz, but my approach was very simplistic, so the tool is not any better than MSAN, it just has completely different limitations.

A more sophisticated approach that's closer to what MSAN is doing might yield better results. Or perhaps someone could just hammer MSAN into something generally applicable.