In notebooks/, we have provided examples in Jupyter Notebooks Using the CWL Evaluation Framework are in the notebooks directory.
- SIGIR2019-Demo-CWL-Plots.ipynb: shows how the CWL framework can be used to inspect the internal continuation, weight and likelihood of stopping vectors
- SIGIR2019-Demo-Measurement-Plots: shows the different measurements from the CWL framework provide different insights into how rankings are scored.
Assumes that the cwl framework is installed.
In compatiability/, we have provided a number of bash scripts that compare the ranking of TREC_EVAL and INST_EVAL against CWL_EVAL.
Assumes that TREC_EVAL and INST_EVAL have been installed.