tommielke / bpmn-miwg-test-suite

BPMN 2.0 Test Cases (Models, Diagrams, Serializations) created by the BPMN Model Interchange Working Group (BPMN MIWG) at the OMG

BPMN Model Interchange Test Suite

BPMN 2.0 Test Cases (Models, Diagrams, Serializations) created by the BPMN Model Interchange Working Group (BPMN MIWG) at the OMG.

For more information see: http://www.omgwiki.org/bpmn-miwg

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

How to test a BPMN tool using this test suite?

Everybody is welcome to test BPMN tools and submit results to the working group, not just tools vendors. Here is how you do it:

1. Download Test Cases

First download the latest version of the BPMN Model Interchange Test Suite and unpack it. The top level directories of the test suite distinguish different categories of tests. Inside each category there is a directory called Reference, which contains the files needed for testing. Next to that, there will be multiple folders with test results of other tools.

2. Test Import, Export and/or Roundtrip of BPMN Models.

There are three different test procedures that can be performed with the provided reference models depending on the capabilities of the tool. You will find explanations of these three procedures below.

3. Submit Your Test Results to BPMN MIWG

Package all result files in a zip file and send it via email to [e-mail]. Make sure to provide the exact name and version of the tool that you tested in the email.

Alternatively, you may also fork the repository on GitHub and submit test results through a pull request.

Note: By submitting test results you agree that they will be published under a Creative Commons Attribution 3.0 Unported License.

Detailed Descrition of Test Procedures

Available Features	Possible Tests
import only	import
export only	export
import & export	import, export & roundtrip

1. BPMN MIWG Import Test Procedure

If the tool under test supports the import of BPMN XML files, you can test that feature using the following procedure:

1. Import the BPMN XML file (*.bpmn) provided with a reference model into the tool, e.g. A.1.0.bpmn.
2. Save the graphical representation of the model as imported to a file named <test id>-import.<file type>, e.g. A.1.0-import.png. You can either do that through an export feature of the tool, e.g. a PNG, JPEG or PDF Export, or by taking a screenshot of the canvas.
3. Compare the result with the image provided with the reference model, e.g. A.1.0.pdf.
4. Report your findings in the spreadsheet BPMN MIWG Test Case Structure.xlsx by putting either OK or Issues into a column labeled with tool name, version and date of test.
5. Create new issues on GitHub – one for each of your findings. Make sure to mention the tool you tested, the id of the test case (e.g. A.1.0) and the type of the test (e.g. import) in the issue title.

Repeat these steps for all reference models.

2. BPMN MIWG Export Test Procedure

If the tool under test supports the export of BPMN XML files, you can test that feature using the following procedure:

1. Draw the model as shown in the image provided with a reference model into the tool, e.g. A.1.0.pdf.
2. Save the graphical representation of the model as drawn to a file named <test id>-export.<file type>, e.g. A.1.0-export.png. You can either do that through an export feature of the tool, e.g. a PNG, JPEG or PDF Export, or by taking a screenshot of the canvas.
3. Export the model into a BPMN XML file (*.bpmn) named <test id>-export.bpmn, e.g. A.1.0-export.bpmn.
4. Compare the result with the BPMN XML file (*.bpmn) provided with the reference model, e.g. A.1.0.bpmn.
5. Report your findings in the spreadsheet BPMN MIWG Test Case Structure.xlsx by putting either OK or Issues into a column labeled with tool name, version and date of test.
6. Create new issues on GitHub – one for each of your findings. Make sure to mention the tool you tested, the id of the test case (e.g. A.1.0) and the type of the test (e.g. export) in the issue title.

Repeat these steps for all reference models.

3. BPMN MIWG Roundtrip Test Procedure (BPMN Import and Export combined)

If the tool under test supports the import and export of BPMN XML files, you can test a roundtrip using the following procedure:

1. Import the BPMN XML file (*.bpmn) provided with a reference model into the tool, e.g. A.1.0.bpmn.
2. If not already done during an import test, save the graphical representation of the model as it is imported to a file named <test id>-import.<file type>, e.g. A.1.0-import.png. You can either do that through an export feature of the tool, e.g. a PNG, JPEG or PDF Export, or by taking a screenshot of the canvas.
3. Export the model into a BPMN XML file (*.bpmn) named <test id>-roundtrip.bpmn, e.g. A.1.0-roundtrip.bpmn.
4. Compare the result with the BPMN XML file (*.bpmn) provided with the reference model, e.g. A.1.0.bpmn.
5. Report your findings in the spreadsheet BPMN MIWG Test Case Structure.xlsx by putting either OK or Issues into a column labeled with tool name, version and date of test.
6. Create new issues on GitHub – one for each of your findings. Make sure to mention the tool you tested, the id of the test case (e.g. A.1.0) and the type of the test (e.g. roundtrip) in the issue title.

Repeat these steps for all reference models.

4. BPMN Import and Export not supported

If the tool does neither support import nor export of BPMN XML files you can not perform the tests described above. However, if the tool supports other model interchange formats, we are still interested to record this information:

1. Open the spreadsheet BPMN MIWG Test Case Structure.xlsx, add a column labeled with tool name, version and date of test, and enter a text like:

The BPMN XML format is not supported. Model Interchange is only possible in the XPDL format. However, this has not been tested by the BPMN MIWG.

Repository Structure

The top level defines test categories. The second level is either a vendor or the submitted reference. The third level holds the individual test case files in the form <identifier>-<test type>.filetype.

The vendor who provided the reference model for a test case will be identified in the summarizing spreadsheet BPMN MIWG Test Case Structure.xlsx.

• README.md (this file)
• BPMN MIWG Test Case Structure.xlsx
• A - Fixed Diagrams with Variations of Attributes
  • Reference <-- reference files for testing
    • A.1.0.png - A.1.0.bpmn <-- look in spreadsheet `BPMN MIWG Test Case Structure.xlsx` to find vendor who provided this reference - ...
  • Business Process Incubator 4.?.? <-- test results
    • A.1.0-import.png - A.1.0-export.png - A.1.0-export.bpmn - A.1.0-roundtrip.bpmn - ...
  • camunda Modeler 2.0.11 <-- test results
    • A.1.0-import.png - A.1.0-roundtrip.bpmn - ...
  • MID Innovator 11.5.1.30223 <-- test results
    • A.1.0-import.png - A.1.0-export.bpmn - ...
• B - Validate that tool covers conformance class set
  • Reference <-- reference files for testing
  • Business Process Incubator 4.?.? <-- test results
  • camunda Modeler 2.0.11 <-- test results
  • ...