Extending benchmarks to Rust targets
addisoncrump opened this issue · comments
As discussed in FUZZING'23, we want to extend Fuzzbench to allow for Rust targets. Not all fuzzers will be compatible, but we would like to define a standard interface for those which are. We suggest the following and ask for input so that future fuzzers may also support Rust benchmarks.
Rust harnesses
Most Rust fuzz harnesses built with cargo-fuzz should be straightforward to modify such that they emit staticlib build units instead of executable binaries. In recent versions of libfuzzer-sys, the common crate for providing libfuzzer bindings to cargo-fuzz targets, you can choose to not link libfuzzer and instead only use the function names. This turns a Rust harness into a classical LLVMFuzzerTestOneInput-style harness. We propose to build Rust benchmarks in this way and link against the static libraries like existing benchmarks.
Build flags and compilers
Rust fuzzers may use a specialised rustc by exporting the CARGO_BUILD_RUSTC environmental variable. In addition, fuzzers using default rustc may export specific build flags with the RUSTFLAGS environmental variable. The latter is done by cargo-fuzz.
Detecting Rust compatibility
If a fuzzer does not set the RUSTFLAGS or CARGO_BUILD_RUSTC variable before performing the build, we should assume that it does not support Rust (no instrumentation passes would be executed). Alternatively, we should export a fuzzer.compatible_languages() function in each fuzzer, which may future-proof in the case of e.g. JavaScript/Java-specific fuzzers.
Hmm...I guess we could also use the blacklisting feature to disable incompatible fuzzers. and it would keep the API thinner. Let me think about this.
I like the idea of having benchmarks opt-in to be part of a "group" (e.g. rust) and then fuzzers can say which groups they want to be fuzz.
I think it makes the most sense if you ever want to extend to e.g. JS/Python/firmware etc. fuzzing.
Can we start with blacklisting fuzzers for now? I'm happy to accept rust targets, though I'm not sure there's real benefit (hard to believe we could write a fuzzer better/worse for rust than C/C++)