XFormTest

XFormTest is a tool for creating and running tests to ensure the quality and stability of XForms.

Quick start

Prerequisites: i. Java 8, ii. access to an XLSForm (excel) to XForm (xml) converter, such as pyxform or XLSForm online, or XLSForm offline.
Download XFormTest.
Download this example XLSForm and convert it to XML.
Run the pre-made tests in the example form (replace "x.y.z" with the version number in the file name of the Java jar file downloaded): java -jar xform-test-x.y.z.jar xlsxExample.xml

Quick start video: https://www.youtube.com/watch?v=doAr26GaTSQ

Installation

XFormTest does not need to be installed. You simply have to have Java 8 in order to run it.

Pre-requisities

Java JDK 8+

Download

You can download XFormTest from the GitHub release page.

Basic Usage

Command syntax

java -jar path/to/xform-test-x.y.z.jar path/to/someFile.xml

Example usage

If you've downloaded version 0.1.0 of XFormTest and an XForm XML file called "someFile.xml", and both of these are in the same folder, you would open up your terminal at that folder and run the following command: java -jar xform-test-0.1.0.jar someFile.xml

Creating tests

XFormTest introduces a concept we call the "linear scenario test". A linear scenario test defined as a set of assertions to be executed in a sequence, one after the other.

XLSForm setup

For a quick idea of how this works in an XLSForm, check out the example below. There's also a demo form available for download here.

An example XLSForm with tests.

Adding the XFormTest column

XFormTest utilizes special columns for linear assertion. Any of the following 3 column names are valid.

bind::xform-test-linearAssert
bind::xform-test
bind::test

Any text entered into the cells of this column are considered "assertions". An "assertion" represents something about the particular question or prompt that you expect to be true. In the context of a linear scenario test, the assertion states something expected to be true not just in isolation, but also true following every other assertion made in the form up to that point.

Adding assertions

There are 2 types of assertions: value assertions, and relevant assertions.

Value assertions indicate that the user is able to enter the given specified value for a given question. In a future release of XFormTest, we are also planning to allow assertions on calculates.

Relevant assertions allow you to simply assert whether a given question/prompt is relevant or not.

Assertion type	XLSForm examples	XForm example
Value	`Bob` or `value: Bob`	`<bind ... xform-test-linearAssert="Bob"/>`
Relevant	`relevant: 0`	`<bind ... xform-test-linearAssert="relevant: 0"/>`

Multiple tests in a single form

Support for multiple linear scenario tests (e.g. multiple columns in a single XLSForm) is not currently available, but planned to be implemented in a future release.

Syntax reference

Expressions

[nodeset attribute-1]: [nodeset attribute-1-evaluated-value], ...

Quick example: relevant: 1, value: yes, constraint: 0

Assertion types

Value

A value assertion is the assertion that some value can be entered for a given question / node. The data type for a value assertion depends on the question / node type. A text type would accept string values, an integer type would accept integer values, a calculate type could accept any kind of data value, and so on.

For example, for a quesiton such as "How many days a week is this facility open?", one might want to value assertion of a number 1 through 7.

Relevant

A relevant assertion is an assertion on the evaluation of an expression set on the relevant attribute of a given question / node. It accepts a boolean of one of the following forms: false, true, 0 or 1.

For example, if there are a particular question has complex pre-conditions, one might want to create a linear scenario test where, after these pre-conditions are met, the relevant of the questions hould evaluate to 1.

Miscellaneous

Useful resources

General resources

XForm Test documentation
XForm Test on GitHub
Open Data Kit - An open source data collection platform utilizing XForms.
PMA2020 - A programmatic data collection organization for rapid-turnaround surveys monitoring key health and development indicators. Birth place of XForm Test.
PMA2020 on GitHub - Open source projects and initiatives by PMA2020.

Other XML and XForm based testing projects

XMLUnit - An XML language native, platform agnostic library for comparing XML documents, schema validation, and XPath value assertion
XForms Unit - a non-implemented unit testing library specification
Orbeon Testsuite - A selenium-based test runner specifically for Orbeon
XForms Client-Side Unit Tests - Another Orbeon-specific test suite for unit tests implemented in JavaScript

Giving feedback or reporting bugs

Send an e-mail to jflack@jhu.edu, or, even better, write an issue on GitHub.

Collaboration

XForm-test is forkable. Pull requests welcome!

Getting help

Send an e-mail to jflack@jhu.edu or write an issue on GitHub. For now, live, real time assistance is also available on Skype. Send a message to: joeflack4.

livinggoods / xform-test