chromy / ink-proof

Testing for Ink compilers and runtimes

ink-proof

Conformance testing for Ink compilers and runtimes

Ink is an open-source narrative scripting language created by Inkle. ink-proof is a tool for conformance testing Ink compilers and runtimes.

Users author interactive stories as .ink files. Inkle provide a compiler (inklecate) which converts these .ink files to a json based format which is then interpreted by a C# runtime. There are other compiler and runtime implementations at various levels of completeness. For example inkjs, godot-ink, inkcpp.

ink-proof consists of a number of .ink and .json test cases. Each test case contains a .ink or .json source file, an input file, and an expected output or "transcript" file. ink-proof runs each test case against every runtime and compiler and compares the actual output to the expected output. Results are generated as a webpage for easy viewing.

The latest public run of ink-proof is available at https://chromy.github.io/ink-proof however you can also run the tool offline as follows:

git clone https://github.com/chromy/ink-proof.git
cd ink-proof
python3 install_deps.py
python3 proof.py --serve
# Now navigate to http://localhost:8000

Test cases

ink-proof has two kinds of test cases: ".ink" test cases and ".json" test cases.

Ink test cases live under the ink directory. They consist of a story.ink file (for example ink/I002/story.ink), an input.txt file, an expected output file named transcript.txt, and metadata in a file named metadata.json.

Json test cases live under the bytecode directory. They consist of a .json file (for example bytecode/B005/story.json), an input.txt file, an expected output file named transcript.txt, and metadata in a file named metadata.json.

Compiler and runtime drivers

ink-proof can test both Ink compilers and Ink runtimes. To do this it uses small shim programs which wrap each compiler or runtime to provide a consistant interface. These shim programs live in the driver directory.

Compiler drivers are named with the suffix _compiler_driver, for example inklecate_v0.9.0_compiler_driver. During testing they are invoked as follows:

your_fancy_compiler_driver -o output.json input.ink

Runtime drivers are named with the suffix _runtime_driver, for example inkjs_v1.9.0_runtime_driver. During testing they are invoked (kind of) as follows:

cat input.txt | your_fancy_runtime_driver story.json >actual_output.txt

In other words they get passed the compiled json story as the only command line argument then they get fed each choice as input and are expected to produce the output on stdout.

This is how inklecate works currently so for other runtimes/compilers this means writing a wrapper to make them work like inklecate.