This repository contains the data files and the utility framework of the BugsJS benchmark.
This benchmark includes bugs from the following 10 projects:
- Bower
- Eslint
- Express
- Hessian.js
- Hexo
- Karma
- Mongoose
- Node-redis
- Pencilblue
- Shields
The tests of the Node-redis project require ipv6 support.
In the Projects directory, there is a subdirectory for each project which contains the following files:
- [project]_bugs.csv: pre-calculated information about the tests of each bug
- [project]_commands.csv: the necessary test runner commands for each bug
- [project]_issues.bin: the bug reports of the subject project save in the Google's Protocol Buffers format (for the structure see the project.proto file)
The framework's command-line interface includes the following commands:
- info: Prints out information about a given bug.
- checkout: Checks-out the source code for a given bug.
- test: Runs all tests for a given bug and measures the test coverage.
- per-test: Runs each test individually and measures the per-test coverage for a given bug.
For the checkout, test, and per-test commands, the user can specify the desired code revision:
- buggy: The parent commit of the revision in which the bug was fixed.
- fixed-only-test-change: A revision containing only the tests introduced in the bug fixing commit, applied to the buggy revision.
- fixed: A revision containing both the cleaned fix and the newly added tests.
Example command:
python3 main.py -p Bower -b 1 -t checkout -v fixed -o output/