SEAFOX - Software Evaluation Automation For XML Supports PLCOpen XML and *.EXP files to be imported
Supported algorithms
- Random
- Basechoice
- Pairwise
A fully integrated tool chain consisting of CODESYS, CfUnit, SEAFOX, and a python script used to combine these, and where the SEAFOX tool was further extended to support additional standard data types. This tool chain can be used to create test cases, generate new input values for these using SEAFOX, automate the additional test case creation and integration with CfUnit in CODESYS IDE, as well as for test execution.
The results of the experiment show that test cases generated by this tool chain achieved on average 90% decision coverage or higher regardless of the combinatorial technique used, with the overall average level being 94%. Interestingly enough, Random testing scored higher than both Pairwise and Base choice testing.
We present our integrated tool chain as a contribution to the automation of test creation and execution for industrial control software in CODESYS IDE. However, we identified several limitations with testing CODESYS timer blocks and when executing larger test suites that can make it problematic to fully test PLC programs in a real-time simulation environment. These challenges should be investigated further by both researchers and practitioners.