Build coverage failed for curl_curl_fuzzer_http
jiradeto opened this issue · comments
just found out that library zlib version 1.2.12 which is used by curl_curl_fuzzer_http is no longer exist. This makes coverage build for this benchmark failed due to download URL is invalid: https://www.zlib.net/zlib-1.2.12.tar.gz
=> ERROR [stage-1 13/13] RUN echo "Run fuzzer_build to build the target" && if [ -z "$debug_builder" ] ; then fuzzer_build; fi 3.4s
------
> [stage-1 13/13] RUN echo "Run fuzzer_build to build the target" && if [ -z "$debug_builder" ] ; then fuzzer_build; fi:
#20 1.048 Run fuzzer_build to build the target
#20 1.126 FUZZ_TARGET = curl_fuzzer_http
#20 1.126 CFLAGS = -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -O3
#20 1.126 CXXFLAGS = -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -stdlib=libc++ -O3
#20 1.136 Building benchmark curl_curl_fuzzer_http with fuzzer coverage
#20 1.138 + ./ossfuzz.sh
#20 1.139 CC: clang
#20 1.139 CXX: clang++
#20 1.139 LIB_FUZZING_ENGINE: /usr/lib/libFuzzer.a
#20 1.139 CFLAGS: -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -O3 -fprofile-instr-generate -fcoverage-mapping -gline-tables-only
#20 1.139 CXXFLAGS: -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument -stdlib=libc++ -O3 -fprofile-instr-generate -fcoverage-mapping -gline-tables-only
#20 1.139 ARCHITECTURE: x86_64
#20 1.139 FUZZ_TARGETS: curl_fuzzer_dict curl_fuzzer_file curl_fuzzer_ftp curl_fuzzer_gopher curl_fuzzer_http curl_fuzzer_https curl_fuzzer_imap curl_fuzzer_ldap curl_fuzzer_mqtt curl_fuzzer_pop3 curl_fuzzer_rtmp curl_fuzzer_rtsp curl_fuzzer_scp curl_fuzzer_sftp curl_fuzzer_smb curl_fuzzer_smtp curl_fuzzer_tftp curl_fuzzer
#20 1.142 + [[ -d /src/curl_fuzzer ]]
#20 1.142 + SCRIPTDIR=/src/curl_fuzzer/scripts
#20 1.142 + DEPENDENCY=zlib
#20 1.142 + DEPENDENCYDIR=/src/zlib
#20 1.142 + INSTALLDIR=/src/curl_install
#20 1.142 + [[ -z /src/zlib ]]
#20 1.142 + [[ ! -d /src/zlib ]]
#20 1.142 + /src/curl_fuzzer/scripts/download_zlib.sh /src/zlib
#20 1.143 + wget https://www.zlib.net/zlib-1.2.12.tar.gz -O /tmp/zlib-1.2.12.tar.gz
#20 1.145 --2022-10-16 06:06:03-- https://www.zlib.net/zlib-1.2.12.tar.gz
#20 1.146 Resolving www.zlib.net (www.zlib.net)... 85.187.148.2
#20 1.313 Connecting to www.zlib.net (www.zlib.net)|85.187.148.2|:443... connected.
#20 1.851 HTTP request sent, awaiting response... 404 Not Found
#20 2.036 2022-10-16 06:06:04 ERROR 404: Not Found.
#20 2.036
#20 2.109 Traceback (most recent call last):
#20 2.109 File "<string>", line 1, in <module>
#20 2.109 File "/src/fuzzers/coverage/fuzzer.py", line 33, in build
#20 2.109 utils.build_benchmark()
#20 2.109 File "/src/fuzzers/utils.py", line 82, in build_benchmark
#20 2.109 subprocess.check_call(['/bin/bash', '-ex', build_script], env=env)
#20 2.109 File "/usr/local/lib/python3.8/subprocess.py", line 364, in check_call
#20 2.109 raise CalledProcessError(retcode, cmd)
#20 2.109 subprocess.CalledProcessError: Command '['/bin/bash', '-ex', '/src/build.sh']' returned non-zero exit status 1.
@jonathanmetzman thanks for taking a look into this however I think it is not solved yet
[+] Building 5.4s (9/12)
=> [internal] load build definition from Dockerfile 0.8s
=> => transferring dockerfile: 1.25kB 0.0s
=> [internal] load .dockerignore 1.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for gcr.io/oss-fuzz-base/base-builder@sha256:1b6a6993690fa947df74ceabbf6a1f89a46d7e4277492addcd45a8525e34be5a 0.5s
=> importing cache manifest from gcr.io/fuzzbench/builders/benchmark/curl_curl_fuzzer_http 0.9s
=> [1/7] FROM gcr.io/oss-fuzz-base/base-builder@sha256:1b6a6993690fa947df74ceabbf6a1f89a46d7e4277492addcd45a8525e34be5a 0.0s
=> [internal] load build context 0.6s
=> => transferring context: 30B 0.0s
=> CACHED [2/7] RUN git clone --depth 1 https://github.com/curl/curl.git /src/curl 0.0s
=> CACHED [3/7] RUN git clone https://github.com/curl/curl-fuzzer /src/curl_fuzzer 0.0s
=> ERROR [4/7] RUN git -C /src/curl_fuzzer checkout dd486c1e5910e722e43c451d4de928ac80f5967d 2.4s
------
> [4/7] RUN git -C /src/curl_fuzzer checkout dd486c1e5910e722e43c451d4de928ac80f5967d:
#8 1.279 fatal: reference is not a tree: dd486c1e5910e722e43c451d4de928ac80f5967d
I got the same error. If I remove the sha256 in the FROM command it works (not sure if this is the proper fix though. Suspect it's something with the old Docker cache referenced in the FROM).
Take it back; it doesn't work!
I think my fix worked it's just that our caching doesn't allow it to be used. I'm fixing that manually now but I guess we have a bit of a design flaw here.