Add GPU support to the local experiment of fuzzbench
GoenitzYs opened this issue · comments
Excuse me. I just tried to add GPU support to the container so that I could test my Neural Network-based fuzzer. I have managed to add GPU support in generated.mk and run_experiment.py, and the test on all target program were just passed using command "make test-run-myfuzzer-all". However, when I just started a local experiment using run_experiment.py, the test failed, and the runner
container seemed just not to add GPU support. Could anyone tell me where the problem might be, and what I could do to continue the experiment? Thanks!
PS: It seemed that the container created by "run_experiment.py" just invoked another container to run the experiment, but I couldn't find where the invoke command is.
@GoenitzYs run_experiment
spins up the dispatcher-container
, copies all of the local data into it (if you have allow-uncommitted-changes set) and then the dispatcher-container
starts by first building all of the target docker's (it reuses your registry, so if you've already done the make test-...
it should be fast), then entering the scheduling loop, where it schedules the experiment containers (or up to as many as you have cores for) and waits for them to finish. Inside that scheduling loop is where the actual experiment containers are launched.
Relevant functions in experiment/scheduler.py
(in callstack order, bottom is the one you want probably)
schedule_loop
schedule
start_trials
_start_trial
gcloud->create_instance
common/gcloud.py
:create_instance
common/gcloud.py
:run_local_instance
However, that simply invokes a startup script which is templated in render_startup_script_template
in scheduler.py
. The template for it is in experiment/resources/runner-startup-script-template.sh
, which contains the actual docker command.
runner-startup-script-template.sh
Hope that helps!
@Lukas-Dresel Thank you for your reply! Actually, I tried several solutions these days with the tips you provide, but I could not solve the problem right now. In detail, the dispatcher
could recognize CUDA now but the runner
still does not work after I added CUDA-related commands into the runner-startup-script-template.sh
. I will keep trying and feedback once I find a feasible approach. Thanks again!
following this thread.
follows: Recently I read a new article received from one of my senior, which proposed a new fuzz benchmark (MLFuzz), the dataset is based on Fuzzbench and Google Test Suit, and supports GPU options. One may try the dataset as one of the solution. :)
follows: Recently I read a new article received from one of my senior, which proposed a new fuzz benchmark (MLFuzz), the dataset is based on Fuzzbench and Google Test Suit, and supports GPU options. One may try the dataset as one of the solution. :)
Hi, thank you very much for proposing this question. I am currently migrating my fuzzer to fuzzbench and facing the same issue with enabling GPU support and cuda. May I ask have you solve this? I understand I need to add a command --gpus all
to runner docker to enable GPUs, but I can't find where the script to modify.