Open Standard for Check Results

[Scott Wilson]

One thing that might be interesting would be a standard format for defining result data from checks.

For example, let's say we have a check that returns if a + b == c. The result type would output the following data (if exported to json).

{
    "check": "APlusBEqualsCCheck",
    "status": "passed",
    "message": "A + B == C",
    "items": [
        {
            "value": [1, 2, 3],
            "type_hint": "a_b_c",
        }
    ],
    "check_duration": 123.456,
    "fix_duration": 0.0,
    "can_fix": false,
    "can_skip": false
}

A use case for this would be if checks ran on the farm, and outputted the Json data to a database for a web dashboard for production to review.

This is similar to the JUnit XML format, but that format does not contain the rich amount of data that the checks framework supports, so could be insufficient. It would be a good idea to see what's already out there and see if it would be better to use a format that already exists, or create a new one.

Lastly, there's still some open questions for the design:

• Should the format be for just the results, or results, checks, checks relationships (due to the scheduler), input data, etc?
• Json, XML, etc? Json is a strong candidate, since it is used everywhere. We would probably want to use a standard for the schema (for example, jsonschema).