This repo is a short benchmark of various parsers.
It was used to produce the performance plot in https://dl.acm.org/citation.cfm?id=3360553.
It uses the Python 3.6 grammar in all cases and can download and parse whole repositories off of GitHub.
The paper shows performance results for parsing the source files in SymPy due to space constraints, however this script will, by default run on all the repositories it's aware of.
The current list of repositories that will be used is set up in repos.py
.
This benchmark assumes that the racket and perl based parsers are set up inside different conda environments and that the ANTLR-generated C++ parser has been built in the antlr/build
subdirectory
.
You can install miniconda in $HOME/miniconda
by running
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
bash miniconda.sh -b -p $HOME/miniconda
rm miniconda.sh
Installing miniconda this way does not modify your path by default.
In order to use this miniconda installation make sure that $HOME/miniconda/bin
is on your path.
Please also make sure that git and cmake are installed somewhere on your system.
Marpa is assumed to be installed within a conda environment called perl
.
If you want to use a version of Marpa installed elsewhere, create a dummy conda environment by running conda create -n perl
.
Derp-3 is assumed to be set up in an environment called racket.
To set up the needed conda environment, after installing miniconda, run conda create -n racket -c conda-forge racket
then run source activate racket
and raco pkg install base math-lib srfi-lib
.
You can return to the base conda environment by running source activate base
.
The ANTLR parser generator has already been used to generate the C++ source files in the antlr directory. To compile them, install CMake and a working C++ compiler then change directory to the antlr subdirectory of this repository and run the commands
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j
Once all this setup is finished, you can run the benchmarks by running python bench.py
in the main repo folder.
It is set up to collect data on various Python repositories on GitHub.
The script there should take care of downloading the git repositories needed and cleaning them up afterward.
Once the benchmarks have run, the plots can be regenerated by running python plot.py
.
Matplotlib and cairo are required to generate the plots.