Automatic testing of small programming assignments.
This project consists of two parts, an programming-language-agnostic driver for testing which invokes teacher-provided script on student's solution and a tool for comparison of Haskell expressions and types based on QuickCheck.
This tool was initially used in the Non-Imperative Programming course on Faculty of Informatics, Masaryk University (FI MU) and is now used in 3 more courses on the same faculty (with different-teacher provided components). It can be used either locally (but that is not very useful) or connected to a testing frontend, such as the frontend of IS MU. In either case, testing requires assignment files.
The work began with a bachelor thesis of Martin Jonáš on FI MU, the code was later extended by Vladimír Štill and is used in the couse from autumn 2014. As of autumn 2018, a complete rewrite (version 3) of Haskell testing framework is going to be used.
There is support for multiple evaluators. The original & residing in this repository is Haskell evaluator: hsExprTest (documentation).
Other evaluators are mostly private to the courses which use them. Documentation for writing evaluators will be added.
For integration with IS MU, you will need a server capable of running a
web server, the exprtest service
(documentation) and your language backends of choice.
For running the service, you can start with the systemd unit
file.
The original frontend is the questionnaire frontend which uses ability of IS to send questionnaire evaluation to an external server.
TODO: doc
The ispol (documentation) is an
alternative frontend/glue for IS MU. It allows students to submit a file to
submission folder which is checked by ispol and evaluated using ExprTest
service. The results are written to student's notebooks.
Questions are stored in question directory (defaults to /var/lib/checker/qdir),
they have extension .q.hs, or other .q.?? extension.
Use GitHub bug reporting for bugs (I'm sure there are some).