The PEtab test suite is a collection of test models for the PEtab parameter estimation data format. It is intended to be used to verify and quantify PEtab-support by developers of tools for model simulation and parameter estimation.
The PEtab test suite can be downloaded from GitHub via
git clone https://github.com/petab-dev/petab_test_suite
The test suite comes with all necessary files pregenerated.
The PEtab test suite comes with a Python package named petabtests in the homonymous subdirectory. It contains Python functions for generating the tests and evaluating results. This can be installed via
cd petab_test_suite
pip3 install -e .
The petabtests/cases subdirectory contains different test suites
for different PEtab versions and model formats.
Each test suite is a collection of enumerated tests.
Each test consists of a single PEtab problem defined in XXXX/_XXXX.yaml
file and the expected result (chi2 value, log-likelihood, simulation
table reference, and tolerances) in XXXX/_XXXX_solution.yaml.
XXXX/XXX.md contains a short description of the respective test
problem that is not relevant for the execution of the test itself.
To evaluate how a tool performs on a given test problem, three metrics are employed: Simulations, chi2 value and log-likelihood. A tool can be said to cover a test problem if any of those values matches the ground truth values up to some tolerance.
The Python package provides convenience functions for evaluation in petabtests/evaluate.py.
-> Overview of passed test cases for different tools supporting PEtab
Contributions to the PEtab test suite are welcome.
To add a new test case, create a new subdirectory in the respective test suite
directory under petabtests/cases/. The subdirectory name should be a
four-digit number, starting with the next available number.
Most files in the test suite are generated automatically. Those start with an
underscore, except for the README.md files. The only files that need to be
created manually are the XXXX/XXXX.py files that contain the script to
generate the test problem and solution files. Their content should be
self-explanatory For adding a new test case, copy XXXX/XXXX.py from an
existing test case and adjust it to the new test case.
All remaining files are generated by the petabtests_create script which
will be available on your $PATH after installing the provided Python library
(see above).