Failed to generate test case

Question

Failed to generate test case

zr950624 opened this issue 4 years ago · comments

Hi,

I set up DIE by following your README and after I execute ./fuzz/script/run.sh ~/ch
I got following logs.

*] Command: timeout 30 node ./fuzz/afl/../TS/esfuzz.js output-1/.cur_input.js output-1/fuzz_inputs 100 385230076 > /dev/null
/home/hacker/DIE/fuzz/TS/base/utils.js:136
            throw new Error("[-] " + msg);
            ^

Error: [-] file doesn't exist: output-1/.cur_input.js
    at Object.assert (/home/hacker/DIE/fuzz/TS/base/utils.js:136:19)
    at new Code (/home/hacker/DIE/fuzz/TS/base/estestcase.js:17:17)
    at main (/home/hacker/DIE/fuzz/TS/esfuzz.js:22:16)
    at Object.<anonymous> (/home/hacker/DIE/fuzz/TS/esfuzz.js:46:1)
    at Module._compile (internal/modules/cjs/loader.js:1137:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
    at Module.load (internal/modules/cjs/loader.js:985:32)
    at Function.Module._load (internal/modules/cjs/loader.js:878:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47

It seems failed to generate new test case.

Park Soyeon · Answer 1 · Sun Aug 02 2020 13:51:25 GMT+0800 (China Standard Time)

.cur_input.js should be created by getNextTestCase command which is executed before the command. (timeout 30 node ./fuzz/afl/../TS/esfuzz.js output-1/.cur_input.js output-1/fuzz_inputs 100 385230076 > /dev/null)

What I can imagine is the command failed probably because of two reasons:

Your machine doesn't make valid connection with redis-server.

Please check the tmux-session named ssh-tunneling.
In this case, you might see some error msg when the command was executed.
Please compare your log with below messages.

[*] Command: node ./fuzz/afl/../TS/redis_ctrl.js getNextTestcase output-0/.cur_input.js
[*] Generating testcases...
[*] Command: timeout 30 node ./fuzz/afl/../TS/esfuzz.js output-0/.cur_input.js output-0/fuzz_inputs 100 1066761907 > /dev/null
[*] Scanning 'output-0/fuzz_inputs'...
[*] Command: node ./fuzz/afl/../TS/redis_ctrl.js reportStatus fuzzer-$(hostname)-$(cat /etc/machine-id|cut -c 1-16)-258243 output-0/fuzzer_stats
[*] Time - Generation: 101.00 ea/s, Execution: 25.25 ea/s

You don't have write permission on the "output-1" directory.

It's unlikely to happen though.

Rui Zhong · Answer 2 · Sun Aug 02 2020 14:39:07 GMT+0800 (China Standard Time)

I saw you forward 6379 port of redis server to 9000 port of fuzzing machine.
In my case, I set up DIE in a single docker, so I patch /fuzz/scripts/run-all.py

diff --git a/fuzz/scripts/run-all.py b/fuzz/scripts/run-all.py
index 00f409d..9d04dda 100755
--- a/fuzz/scripts/run-all.py
+++ b/fuzz/scripts/run-all.py
@@ -4,15 +4,15 @@ import os
 import multiprocessing
 from os import environ

-environ["REDIS_URL"] = "redis://localhost:9000"
+environ["REDIS_URL"] = "redis://localhost:6379"


 if __name__ == '__main__':
     p = argparse.ArgumentParser()
     p.add_argument('cmd', nargs='+')
-    p.add_argument('--cpu', nargs='?', type=int, default=int(multiprocessing.cpu_count() - 4))
+    p.add_argument('--cpu', nargs='?', type=int, default=5)
     cmd = p.parse_args().cmd
     assert('output' in cmd)
     for i in range(p.parse_args().cpu):
         new_cmd = ' '.join(cmd).replace('output', 'output-%d' % i)
-        os.system('tmux new-window -n jsfuzz-%d "AFL_NO_UI=1 REDIS_URL=redis://localhost:9000 %s; /bin/bash"' % (i, new_cmd))
+        os.system('tmux new-window -n jsfuzz-%d "AFL_NO_UI=1 REDIS_URL=redis://localhost:6379 %s; /bin/bash"' % (i, new_cmd))

I think this patch can solve the redis connection so I do not need to execute fuzz/script/redis.py anymore. (I'm not sure)
After running ./fuzz/scripts/populate.sh ~/ch, the redis seems good for fuzzing.

444cbf804565:~/DIE$ redis-cli
127.0.0.1:6379> keys *
 1) "fuzzers:fuzzer-444cbf804565--107881"
 2) "fuzzers:fuzzer-444cbf804565--80712"
 3) "fuzzers:fuzzer-444cbf804565--35328"
 4) "fuzzers:fuzzer-444cbf804565--24680"
 5) "fuzzers:fuzzer-444cbf804565--93271"
 6) "fuzzers:fuzzer-444cbf804565--93286"
 7) "fuzzers:fuzzer-444cbf804565--35338"
 8) "fuzzers:fuzzer-444cbf804565--24666"
 9) "fuzzers:fuzzer-444cbf804565--107886"
10) "fuzzers:fuzzer-444cbf804565--35323"
11) "fuzzers:fuzzer-444cbf804565--93281"
12) "fuzzers"
13) "fuzzers:fuzzer-444cbf804565--107891"
14) "fuzzers:fuzzer-444cbf804565--24661"
15) "fuzzers:fuzzer-444cbf804565--24670"
16) "fuzzers:fuzzer-444cbf804565--37622"
17) "fuzzers:fuzzer-444cbf804565--80717"
18) "fuzzers:fuzzer-444cbf804565--37607"
19) "fuzzers:fuzzer-444cbf804565--35343"
20) "fuzzers:fuzzer-444cbf804565--80708"
21) "fuzzers:fuzzer-444cbf804565--107876"
22) "fuzzers:fuzzer-444cbf804565--107871"
23) "fuzzers:fuzzer-444cbf804565--37627"
24) "fuzzers:fuzzer-444cbf804565--93276"
25) "fuzzers:fuzzer-444cbf804565--80727"
26) "fuzzers:fuzzer-444cbf804565--37612"
27) "fuzzers:fuzzer-444cbf804565--35334"
28) "fuzzers:fuzzer-444cbf804565--24675"
29) "fuzzers:fuzzer-444cbf804565--37617"
30) "fuzzers:fuzzer-444cbf804565--93291"
31) "fuzzers:fuzzer-444cbf804565--80722"
127.0.0.1:6379>

I also check theoutput-n directory is writable.
Then I execute run.sh

[*] Get a next testcase
[*] Command: node ./fuzz/afl/../TS/redis_ctrl.js getNextTestcase output-1/.cur_input.js
[-] getNextTestcase - Need to populate first
[*] Generating testcases...
[*] Command: timeout 30 node ./fuzz/afl/../TS/esfuzz.js output-1/.cur_input.js output-1/fuzz_inputs 100 809608704 > /dev/null
/home/hacker/DIE/fuzz/TS/base/utils.js:136
            throw new Error("[-] " + msg);
            ^

Error: [-] file doesn't exist: output-1/.cur_input.js
    at Object.assert (/home/hacker/DIE/fuzz/TS/base/utils.js:136:19)
    at new Code (/home/hacker/DIE/fuzz/TS/base/estestcase.js:17:17)
    at main (/home/hacker/DIE/fuzz/TS/esfuzz.js:22:16)
    at Object.<anonymous> (/home/hacker/DIE/fuzz/TS/esfuzz.js:46:1)
    at Module._compile (internal/modules/cjs/loader.js:1137:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
    at Module.load (internal/modules/cjs/loader.js:985:32)
    at Function.Module._load (internal/modules/cjs/loader.js:878:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47
[*] Scanning 'output-1/fuzz_inputs'...
[*] Time - Generation: inf ea/s, Execution: inf ea/s

Is this situation caused by [-] getNextTestcase - Need to populate first?

Park Soyeon · Answer 3 · Mon Aug 03 2020 02:44:15 GMT+0800 (China Standard Time)

From the error msg and redis's status, it seems the population was not well done.

It mostly happens because of environment setup related issues of AFL.

To debug it, You can check the stdout in tmux session named corpus after executing ./fuzz/scripts/populate.sh ~/ch.

If it's executed well, the redis server should contain "crashBitmap", "crashQueue", "pathBitmap", "newPathsQueue".

Rui Zhong · Answer 4 · Mon Aug 03 2020 03:38:25 GMT+0800 (China Standard Time)

The output of tmux session corpus attached as following

afl-fuzz 2.52b by <lcamtuf@google.com>                                                                                                                                                             [391/391]
[+] Disabling the UI because AFL_NO_UI is set.
[+] You have 48 CPU cores and 12 runnable tasks (utilization: 25%).
[+] Try parallel jobs - see docs/parallel_fuzzing.txt.
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #0.
[*] Checking core_pattern...
[*] Checking CPU scaling governor...
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[+] No auto-generated dictionary tokens to reuse.
[*] Validating target binary...

[+] Here are some useful stats:

    Test case count : 0 favored, 0 variable, 0 total
       Bitmap range : 0 to 0 bits (average: 0.00 bits)
        Exec timing : 0 to 0 us (average: 0 us)

[*] No -t option specified, so I'll use exec timeout of 1000 ms.
[+] All set and ready to roll!
[*] Command: node ./fuzz/afl/../TS/redis_ctrl.js reportStatus fuzzer-$(hostname)-$(cat /etc/machine-id|cut -c 1-16)-35323 output-0/fuzzer_stats
[*] Scanning './corpus/output-0'...
[*] Checking corpus: ./corpus/output-0/000000-corpus.js
[*] Spinning up the fork server...
[+] All right - fork server is up.
[*] Command: node ./fuzz/afl/../TS/redis_ctrl.js reportStatus fuzzer-$(hostname)-$(cat /etc/machine-id|cut -c 1-16)-35323 output-0/fuzzer_stats
[*] Checking corpus: ./corpus/output-0/000001-corpus.js
[*] Checking corpus: ./corpus/output-0/000002-corpus.js
[*] Checking corpus: ./corpus/output-0/000003-corpus.js
[*] Checking corpus: ./corpus/output-0/000004-corpus.js
[*] Checking corpus: ./corpus/output-0/000005-corpus.js
[*] Checking corpus: ./corpus/output-0/000006-corpus.js
[*] Checking corpus: ./corpus/output-0/000007-corpus.js
[*] Checking corpus: ./corpus/output-0/000008-corpus.js
[*] Checking corpus: ./corpus/output-0/000009-corpus.js
[*] Checking corpus: ./corpus/output-0/000010-corpus.js
[*] Checking corpus: ./corpus/output-0/000011-corpus.js
[*] Checking corpus: ./corpus/output-0/000012-corpus.js
[*] Checking corpus: ./corpus/output-0/000013-corpus.js
[*] Checking corpus: ./corpus/output-0/000014-corpus.js
[*] Checking corpus: ./corpus/output-0/000015-corpus.js
[*] Checking corpus: ./corpus/output-0/000016-corpus.js
[*] Checking corpus: ./corpus/output-0/000017-corpus.js
[*] Checking corpus: ./corpus/output-0/000018-corpus.js
[*] Checking corpus: ./corpus/output-0/000019-corpus.js
[*] Checking corpus: ./corpus/output-0/000020-corpus.js
[*] Checking corpus: ./corpus/output-0/000021-corpus.js
....
....


+++ Testing aborted by user +++
[+] We're done here. Have a nice day!

I didn't see "crashBitmap", "crashQueue", "pathBitmap" or "newPathsQueue" in this output.
Only a lot of "Checking corpus" and a command "reportStatus" which is called from fuzz/afl/afl-fuzz.c#L3641

u8* cmdline = alloc_printf(
      "node %s/../TS/redis_ctrl.js reportStatus "
      "fuzzer-$(hostname)-$(cat /etc/machine-id|cut -c 1-16)-%d %s",
      own_loc, getpid(), fn);

I instrumented the binary with a clean AFL, will it cause any problem?

It seems after calling fuzz_dir, it directly exits fuzzing, so there is no chance to call save_if_interesting.

if (in_dir) {
    fuzz_dir(in_dir, use_argv);
    goto stop_fuzzing;
  }

Rui Zhong · Answer 5 · Mon Aug 03 2020 04:33:20 GMT+0800 (China Standard Time)

It seems we need to add -C while populating test cases.
After add -C in populate.sh, it works well.

Park Soyeon · Answer 6 · Mon Aug 03 2020 04:38:52 GMT+0800 (China Standard Time)

Is your ~/ch well-instrumented with provided afl? I realized I forgot to add how to compile the target binary with afl-clang-fast.
populate.sh works without -C in my environment so I concern that it's not the root cause.

Rui Zhong · Answer 7 · Mon Aug 03 2020 04:42:33 GMT+0800 (China Standard Time)

I compile with the original AFL, did you modify any code in afl-clang-fast?

Park Soyeon · Answer 8 · Mon Aug 03 2020 05:02:01 GMT+0800 (China Standard Time)

afl-llvm-pass.so.cc has minor change but it might not matter if your target is working well with original AFL.

wangkun123-art · Answer 9 · Wed Jan 27 2021 14:50:13 GMT+0800 (China Standard Time)

can you add the steps to get ~/ch ? only like AFL?

Kakaro_bot · Answer 10 · Fri Jul 08 2022 22:53:24 GMT+0800 (China Standard Time)

Hi. Have you solved all problems? I have so many errors, which i wish you can help me. pls~~~