Testability of normative DID Spec Registries statement on metadata
msporny opened this issue · comments
@shigeya wrote:
Let me quote the statement again here:
All metadata property definitions registered in the DID Specification Registries [DID-SPEC-REGISTRIES] MUST define the value type, including any additional formats or restrictions to that value (for example, a string formatted as a date or as a decimal integer).
From the core spec point of view, the above statement is not testable within the core spec. This spec text needs to be placed in DID-SPEC-REGISTRIES, not within core spec. I honestly have no idea whether we can treat this statement as "human testable."
From a different perspective: since the current test implementation runs the whole test against the transcript of interaction with multiple method implementations, it is desirable to test properties defined in DID-SPEC-REGISTRIES. In this case, we need a machine-readable -- at least partial -- DID-SPEC-REGISTRY.
I think there are three options:
- Eliminating the above statement
- Rewrite the above statement to move the test to DID-SPEC-REGISTRIES
- Rewrite the above statement to remove (direct) dependency on DID-SPEC-REGISTRIES.
It is possible to create an optional configuration parameter that may hold machine-readable (possibly partial) DID-SPEC-REGISTRIES, which can be usable for the above option 3.
(Note: due to schedule conflict, I can't attend the special topic call this week)
From the core spec point of view, the above statement is not testable within the core spec.
Correct.
This spec text needs to be placed in DID-SPEC-REGISTRIES, not within core spec.
Yes, agreed.
I honestly have no idea whether we can treat this statement as "human testable."
It is human testable. A human can check to see if "All metadata property definitions registered in the DID Specification Registries [DID-SPEC-REGISTRIES] define the value type, including any additional formats or restrictions to that value (for example, a string formatted as a date or as a decimal integer)."
I suggest we do the second option (if we don't have to go through another CR to do it), or just flag the statement as human testable and not something the WG intended to test. We have lots of these sorts of statements in the DID Methods section.
I suggest we do the second option (if we don't have to go through another CR to do it), or just flag the statement as human testable and not something the WG intended to test.
Fully agreed.
We have lots of these sorts of statements in the DID Methods section.
I will review them.
When I look for the word DID-SPEC-REGISTRIES in did-core-test-suite, there are only two files with the word. That's a very small number. Also, one of the file is on metadata which I'm working on.
From the point of view of completeness of the test, and also for creation of a report, I think it is better to include human testable statements in the test suite as a todo() with some easy-to distinguish prefix ( [Require human test] or similar? ).
For example, in Section 5.2. Verification Method, the following statements are missing as test statements.
A verification method MAY include additional properties. Verification methods SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
In order to maximize global interoperability, the verification method type SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
In Section 5.3:
In order to maximize global interoperability, any such properties used SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
In Section 5.4:
In order to maximize interoperability, the service type and its associated properties SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
etc.
From the point of view of completeness of the test, and also for creation of a report, I think it is better to include human testable statements in the test suite as a todo() with some easy-to distinguish prefix ( [Require human test] or similar? ).
At this point, this isn't done, and is unlikely to be done before the cut off on Tuesday. I suggest we don't do this (or could do it later). Take a look at issue #72 - #76 -- we are just not adding marks to the checkboxes that are associated the human-readable tests during the audit.
@msporny thanks. I understand the plan.
From the consistency point of view, do you want to me to remove the human testable statements (currently marked todo()) from the test?
From the consistency point of view, do you want to me to remove the human testable statements (currently marked
todo()) from the test?
Yes, at this point we should just remove all .todo() statements from the test suite. I don't think they're being used by anyone.
@msporny Created a PR for 7.3.
Found that the following in is still a todo in did-url-dereferencing.js:
it.todo('3.2.2 Relative DID URLs - When resolving a relative DID URL reference, the algorithm specified in RFC3986 Section 5: Reference Resolution MUST be used.');