Treat 501 errors as neither pass nor fail
kjetilk opened this issue · comments
Following CommunitySolidServer/CommunitySolidServer#798 I think it makes sense to treat 501 as neither pass nor fail, as it is the server explicitly acknowledging that it knows it hasn't implemented a feature.
Earl doesn't have the exact right predicate for this, as the test has been done, and it applies, it just doesn't count towards the server pass or failure counts.
We have limited outcomes in the EARL vocab so may need to use earl:failed but add rdfs:comment to highlight the distinction. When creating the report assertions we need to check the status or responseStatus for 501