amyzliao / zombie-coverage

For generating line coverage statistics of each test for the zombie GTP benchmark.

coverage of zombie benchmark

for bangyen's very cool research project

instructions for use

Dependencies:

• Racket
• raco
• raco cover

1. CONFIGURE files-cover/zombie-combined-all.rlt: put all the tests you want to run in there, and the program that the tests are supposed to cover. there should be a (run-tests ...) statement for each test, and each statement should be on a different line. make sure all relative paths are from POV of the root folder, not files-cover folder (see relative path for zombie-hist)
2. RUN cover-script.rkt
3. make sure you are cd into the root folder
4. run the raco cover command from the root folder. you will find the command in files-cover/cover-cmd.txt
5. now you can run the html parser on the files in coverage-new/files-cover!

whats in this repo?

A few things are included in this repo:

1. All the zombie benchmark files (UNTYPED) + their dependencies in files-benchmark and base
2. All the zombie tests in files-tests
3. Files related to determining each test's coverage of zombie in files-cover
   • zombie-combined-all.rkt contains the content of all zombie benchmark files and all zombie tests, combined into one file. note that there are many run-test statements throughout zombie-combined-all.rkt.
   • each run-test-XXX.rkt file is a copy of zombie-combined-all.rkt, but with all but ONE of the run-test statements in zombie-combined-all.rkt removed. the XXX corresponds to the line number of the run-test statement from zombie-combined-all.rkt that is included in run-test-XXX.rkt. the run-test statement can be found at the bottom of the run-test-XXX.rkt file.
   • cover-cmd.txt provides the text of the raco cover command you should run to generate coverage info of all the run-test-XXX.rkt files
4. cover-script.rkt, a script that uses zombie-combined-all.rkt to generate a run-test-XXX.rkt file for each run-test statement in zombie-combined-all.rkt. this script also generates the raco cover command in cover-cmd.txt for ease of running raco cover.
5. coverage-new contains the output of running the raco cover command. open index.html in your browser to see the stats generated by raco cover for each run-test-XXX.rkt file.