clang error upon running make for building cryptofuzz

[RonaldKoornneef81]

Hi, we are stumbling on the following issue.

We followed the steps to build cryptofuzz

• https://github.com/guidovranken/cryptofuzz/blob/master/docs/building.md
• We had to add coreutils and boost lib to make that work
• The python script gen_repository creates 2 files: repository_map.h and repository_tbl.h
• Ran the 3 export command: export CFLAGS="-fsanitize=address,undefined,fuzzer-no-link -O2 -g"
  export CXXFLAGS="-fsanitize=address,undefined,fuzzer-no-link -D_GLIBCXX_DEBUG -O2 -g"
  export LIBFUZZER_LINK="-fsanitize=fuzzer"

On the first attempt of running make, the project builds but with the error:
ld: file not found: /Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/lib/darwin/libclang_rt.fuzzer_osx.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)

After manually updating the llvm and clang version to include the libclang_rt.fuzzer.osx.a and running the steps again, the following error is thrown: ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Are there certain dependencies that we are overlooking?

Kind regards,
Ronald

[Guido Vranken]

Hi Ronald,

Did you also set the env variables CC to clang and CXX to clang++ before compiling? I'm not familiar with this error but https://stackoverflow.com/questions/19821041/g-ld-symbols-not-found-for-architecture-x86-64 indicates that this could be due to trying to compile C++ code with the C compiler.

[RonaldKoornneef81]

Hi Guido,

Thanks for the quick reply. After just installing the latest llvm from scratch the build process seem to work. All *.o files are created, the apps generate_corpus and generate_dict are created.

I was expecting an executable called cryptofuzz, but that does not seem to be created.

when running cryptofuzz from makefile:

cryptofuzz : driver.o executor.o util.o entry.o tests.o operation.o datasource.o repository.o options.o components.o wycheproof.o crypto.o expmod.o mutator.o z3.o numbers.o mutatorpool.o ecc_diff_fuzzer_importer.o ecc_diff_fuzzer_exporter.o botan_importer.o openssl_importer.o builtin_tests_importer.o third_party/cpu_features/build/libcpu_features.a test $(LIBFUZZER_LINK) $(CXX) $(CXXFLAGS) driver.o executor.o util.o entry.o tests.o operation.o datasource.o repository.o options.o components.o wycheproof.o crypto.o expmod.o mutator.o z3.o numbers.o mutatorpool.o ecc_diff_fuzzer_importer.o ecc_diff_fuzzer_exporter.o botan_importer.o openssl_importer.o builtin_tests_importer.o $(shell find modules -type f -name module.a) $(LIBFUZZER_LINK) third_party/cpu_features/build/libcpu_features.a $(LINK_FLAGS) -o cryptofuzz
it just prints
test
make: *** [cryptofuzz] Error 1

Also when running ./generate_corpus /path/to/directory I receiving a segmentation fault.

This is the first time I am working with fuzzers, so not sure where these issues come from.

[Guido Vranken]

Are you sure LIBFUZZER_LINK is set to -fsanitize=fuzzer?

generate_corpus was useful in the past but Cryptofuzz is now capable of generating useful inputs itself via its custom mutator. So you don't need to use it per se. But I will check if there's anything wrong with it.

[Guido Vranken]

Are you at liberty to tell which library you are going to fuzz? If so I can give some useful tips.

[RonaldKoornneef81]

I run this when starting from scratch:

#!/bin/bash
set -euo pipefail

git clone https://github.com/guidovranken/cryptofuzz.git
cd cryptofuzz
python3 gen_repository.py

export CFLAGS="-fsanitize=address,undefined,fuzzer-no-link -O2 -g"
export CXXFLAGS="-fsanitize=address,undefined,fuzzer-no-link -D_GLIBCXX_DEBUG -O2 -g"
export LIBFUZZER_LINK="-fsanitize=fuzzer"

then I run make (attached log file)
build.log

I noticed on this last run that the previous error is back again (ld: file not found: /Library/Developer/CommandLineTools/usr/lib/clang/14.0.0/lib/darwin/libclang_rt.fuzzer_osx.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)) So I'll need to fix that first by having clang15 (which does include the .fuzzer files).
running make clean and rebuilding does not always seem to generate this error

We are a research center for Cryptography, so I would like to run cryptofuzz to test our own internal libs, but unfortunately I cannot share them.

[Guido Vranken]

Ok. If you have trouble acquiring libFuzzer, you can also take https://github.com/llvm/llvm-project/tree/main/compiler-rt/lib/fuzzer and type ./build.sh. That should produce libFuzzer.a. You can then set LIBFUZZER_LINK to the full path of libFuzzer.a and try again.

[RonaldKoornneef81]

Thanks I will try that approach

[RonaldKoornneef81]

Unfortunately that approach causes different issues.

The build.sh from libfuzzer generates an error (see log), the libFuzzer.a files does get created.
libfuzzer.log

Following with the make on cryptofuzz, comes to the same situation where the cryptoFuzz executable is not created and the clang error comes up again:

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
build.log

[RonaldKoornneef81]

Hi,

I got cryptofuzz working, I decided to drop the mac altogether and build a docker image with Ubuntu, The build and compiling of cryptofuzz goes without issues there. I'm thinking that applaclang might have something to do with it, but can't investigate that now.

Cheered a bit to soon, I see the cryptofuzz executable being created but it errors out again with: undefined reference to `typeinfo for cryptofuzz::module::OpenSSL'

Here is a full log:

build.log

[Guido Vranken]

Glad you got it working so far. To resolve that error, add -DCRYPTOFUZZ_NO_OPENSSL to CXXFLAGS, rebuild entry.o and then cryptofuzz, e.g. from the cryptofuzz/ directory:

export CXXFLAGS="$CXXFLAGS -DCRYPTOFUZZ_NO_OPENSSL"
rm entry.o
make

[RonaldKoornneef81]

Yeah, I read that on SO and gave it a try. It results in a fatal error due to too many errors in ./repository_map.h

like this:

./repository_map.h:607:140: error: no member named 'nullopt' in namespace 'std'
{fuzzing::datasource::ID("Cryptofuzz/Digest/BLAKE2S128"), {fuzzing::datasource::ID("Cryptofuzz/Digest/BLAKE2S128"), "BLAKE2S128", std::nullopt} },

[Guido Vranken]

Does this error occur in the Ubuntu build environment? Which .cpp file is including ./repository_map.h? Try adding #include <optional> to that file and try again.

You can also try to compile this test program:

#include <optional>

int main(void)
{
	std::optional<int> x;
	return 0;
}

using

clang++ -std=c++17 x.cpp

And see if that works. If it doesn't, your clang version or stdlibc++ might be too old.

If you can't get this working you can also grab https://github.com/google/oss-fuzz and use an existing Cryptofuzz-based project, like libecc.

git clone --depth 1 https://github.com/google/oss-fuzz
cd oss-fuzz/
infra/helper.py build_fuzzers libecc

This should certainly succeed. The build scripts are in the projects/libecc directory and you can adjust them to your own needs.

[RonaldKoornneef81]

Thanks, I'll investigate those approaches and see if I can make it work.

• I'm running in docker image ubuntu:22.04
• repository_map.h is created by running the python script: gen_repository.py (using python 3) and it does not have any #include
• Ubuntu clang version 14.0.0-1ubuntu1
• libstdc++.so.6 (libc6,x86-64) => /lib/x86_64-linux-gnu/libstdc++.so.6
  libstdc++.so.6 (libc6) => /lib32/libstdc++.so.6

adding #include to repository_map.h prior to running make seemed to have compiled without errors. So it seems it's ready to be used now.

Just to be sure. In order to start fuzzing our lib, I just use command ./cryptofuzz path/to/lib with flags I want from ./cryptofuzz -help=1 or is it done during compiling of our libs I add these?

[Guido Vranken]

If you're adding a new library, you need to implement a harness in a .cpp file. You can copy any of the directories in https://github.com/guidovranken/cryptofuzz/tree/master/modules and work from there. Unfortunately the tool cannot automatically work with libraries for which no harness has been implemented.

[RonaldKoornneef81]

Ok, thanks a lot for the information, especially during the weekend :)

I'm just a "simple" QA, so it seems some development from our side has to be done to create a harness for our libraries, but looking forward to getting that to work.

Thanks again. I'll close this issue now, since the issue I encountered is resolved.