Fail to build bloaty benchmark
jiradeto opened this issue · comments
Hi, I got the following error when starting an experiment on bloaty_fuzz_target. Could you please help to check?
#20 57.15 [262/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/protostream_objectwriter.cc.o
#20 56.14 [253/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/default_value_objectwriter.cc.o
#20 56.25 [254/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/error_listener.cc.o
#20 56.44 [255/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/field_mask_utility.cc.o
#20 56.57 [256/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/json_escaping.cc.o
#20 56.68 [257/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/json_objectwriter.cc.o
#20 56.71 [258/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/json_stream_parser.cc.o
#20 56.85 [259/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/object_writer.cc.o
#20 57.05 [260/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/proto_writer.cc.o
#20 57.14 [261/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/protostream_objectsource.cc.o
#20 57.15 [262/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/protostream_objectwriter.cc.o
#20 57.18 [263/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/type_info.cc.o
#20 57.37 [264/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/type_info_test_helper.cc.o
#20 57.45 [265/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/internal/utility.cc.o
#20 57.60 [266/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/json_util.cc.o
#20 57.61 [267/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/message_differencer.cc.o
#20 57.86 [268/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/time_util.cc.o
#20 58.23 [269/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/util/type_resolver_util.cc.o
#20 58.34 [270/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wire_format.cc.o
#20 58.47 [271/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wrappers.pb.cc.o
#20 58.64 [272/356] Building CXX object third_party/protobuf/cmake/CMakeFiles/protoc.dir/__/src/google/protobuf/compiler/main.cc.o
#20 58.74 [273/356] Building C object third_party/zlib/CMakeFiles/minigzip.dir/test/minigzip.o
#20 58.77 FAILED: /usr/local/bin/clang -DUSE_RE2 -D_LARGEFILE64_SOURCE=1 -D_LIBCXXABI_FUNC_VIS="" -I/src/bloaty/third_party/re2 -I/src/bloaty/third_party/capstone/include -I/work/third_party/zlib -I/src/bloaty -isystem /src/bloaty/third_party/protobuf/src -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -O3 -fsanitize-coverage=trace-pc-guard -O2 -g -DNDEBUG -MD -MT third_party/zlib/CMakeFiles/minigzip.dir/test/minigzip.o -MF third_party/zlib/CMakeFiles/minigzip.dir/test/minigzip.o.d -o third_party/zlib/CMakeFiles/minigzip.dir/test/minigzip.o -c /src/bloaty/third_party/zlib/test/minigzip.c
#20 58.77 /src/bloaty/third_party/zlib/test/minigzip.c:18:10: fatal error: 'zlib.h' file not found
#20 58.77 #include "zlib.h"
#20 58.77 ^~~~~~~~
#20 58.77 1 error generated.
#20 69.61 ninja: build stopped: subcommand failed.
#20 69.61 Traceback (most recent call last):
#20 69.61 File "<string>", line 1, in <module>
#20 69.61 File "/src/fuzzers/afl/fuzzer.py", line 40, in build
#20 69.61 utils.build_benchmark()
#20 69.61 File "/src/fuzzers/utils.py", line 82, in build_benchmark
#20 69.61 subprocess.check_call(['/bin/bash', '-ex', build_script], env=env)
#20 69.61 File "/usr/local/lib/python3.8/subprocess.py", line 364, in check_call
#20 69.61 raise CalledProcessError(retcode, cmd)
#20 69.61 subprocess.CalledProcessError: Command '['/bin/bash', '-ex', '/src/build.sh']' returned non-zero exit status 1.
We're in the middle of upgrading everything in fuzzbench to Ubuntu 20.04. #1541
Not sure we will fix this in the meantime, but i'd probably accept a patch.
Yep, we fixed bloaty in this commit 7330b from the PR Jonathan mentioned.
Thanks for the answer! While waiting for your PR I can pin to previous certain image digests that were not affected by your upgrade, right?
Is it gcr.io/fuzzbench/base-image that caused this break?
Thanks for the answer! While waiting for your PR I can pin to previous certain image digests that were not affected by your upgrade, right?
we haven't done the upgrade yet so I don't know why this is breaking. I don't think going to an older image will fix this.
confusingly gcr.io/fuzzbench/base-image doesn't affect the build, only gcr.io/oss-fuzz-base/base-builder does
I think if you can't wait, you should try to fix the issue by patching whatever is making bloaty fail to find zlib. Sorry we can't help with this but were prioritizing the upgrade.
we haven't done the upgrade yet so I don't know why this is breaking. I don't think going to an older image will fix this.
confusingly gcr.io/fuzzbench/base-image doesn't affect the build, only gcr.io/oss-fuzz-base/base-builder does
Doesn't fuzzbench always pull latest tag image every time we start experiment? I encountered this break a couple days ago
on my fuzzbench branch (not synced with fuzzbench master) and also I see the base image were updated lately . Perhaps this is the case?
I think if you can't wait, you should try to fix the issue by patching whatever is making bloaty fail to find zlib. Sorry we can't help with this but were prioritizing the upgrade.
Yeah sure thing, I fixed on my local and able to start an experiment but just curious what caused this break :)
Doesn't fuzzbench always pull latest tag image every time we start experiment? I encountered this break a couple days ago
Hmm...actually I forgot we use buildkit which does this
Yeah there was an update recently (I think we do it daily to speed up CI). Maybe this approach could work.
Just double-checked that this has been fixed : )
make -j test-run-libfuzzer-bloaty_fuzz_target runs successfully on the master branch.
just curious what caused this break :)
@jiradeto, here is a quick explanation, in case you are still curious about the cause:
As the error message indicates, it is due to missing zlib, and is fixed by installing libz-dev : )