State of the test server

[Bernhard Fuchs]

We would like to run the testsuite as a hosted service at some point. The docker image is running and according to the docs should be callable with this curl:

curl -s -X POST http://localhost:8080/test-suite-manager/generate-report \
-H "Content-Type: application/json" \
-d @./suites/did-spec/default.json \
| jq '.suitesReportJson[0]'

These default.json files got replaced by .js files with PR #52 and replacing them with e.g. -d @./suites/did-resolution/default.js returns following error:

{"statusCode":400,"code":"FST_ERR_CTP_EMPTY_JSON_BODY","error":"Bad Request","message":"Body cannot be empty when content-type is set to 'application/json'"}

We also tried it with suitesInput.json's content as data

[
  {
    "name": "did-resolution"
  }
]

and could run tests but got the error

{
  "error": "Cannot read property 'ancestorTitles' of undefined"
}

The testserver logs following:

did-core-test-server_1  |   ● Test suite failed to run
did-core-test-server_1  |
did-core-test-server_1  |     TypeError: Cannot read property 'forEach' of undefined
did-core-test-server_1  |
did-core-test-server_1  |        7 | const utils = require('../resolution-utils');
did-core-test-server_1  |        8 |
did-core-test-server_1  |     >  9 | suiteConfig.resolvers.forEach((implementation) => {
did-core-test-server_1  |          |                       ^
did-core-test-server_1  |       10 |   let implementationName = `7.1.x DID Resolution - ${implementation.implementation} - ${implementation.implementer}`;
did-core-test-server_1  |       11 |
did-core-test-server_1  |       12 |   describe(implementationName, () => {
did-core-test-server_1  |
did-core-test-server_1  |       at Object.<anonymous> (packages/did-core-test-server/suites/did-resolution/did-resolution.spec.js:9:23)
did-core-test-server_1  |
did-core-test-server_1  | Test Suites: 1 failed, 1 total
did-core-test-server_1  | Tests:       0 total
did-core-test-server_1  | Snapshots:   0 total
did-core-test-server_1  | Time:        0.317 s
did-core-test-server_1  | Ran all test suites.

Our attempts to add implementations as input failed so far.

Is this endpoint working atm? If so pls give some example inputs.

[Orie Steele]

This issue seem to be blocking a significant PR and should be addessed ASAP.

from what I can tell, there is stale JSON in the repo that needs to be updated to reflect the current JSON model assumed by the suites.

I suggest starting from a small example that passes, and adding json one by one....

We might also dump the JSON that is currently passing to disk and use it as input to resolve this.

[Bernhard Fuchs]

Sorry for the confusion but this issue is not blocking #96.
The implementation files can be merged and the testsuite can be run locally.

For us the next step would be to host the testserver somewhere (maybe at the Universal Resolver cluster), so we have an endpoint against we can run the files/inputs from above and save the time of going through a PR and run the testuite manually.

[Orie Steele]

@BernhardFuchs awesome, thats certainly possible to do, or should be... my guess is the summary i gave is still accurate, the json in the repo is stale, and needs to be cleaned up.... and we probably should publish a docker container that can make testing easier for everyone.

[caict-develop-zhangbo]

I have the same problem. Is there any solution now？

[Bernhard Fuchs]

@caict-develop-zhangbo Currently I'm working on this but will take a few more days. Once I have a draft, I will open a PR and mention it in this issue.

[caict-develop-zhangbo]

@caict-develop-zhangbo Currently I'm working on this but will take a few more days. Once I have a draft, I will open a PR and mention it in this issue.

@BernhardFuchs Looking forward to your PR. And by the way, I found " require('../implementations/universal-resolver-did-factom.json')" in suites/did-resolution/default.js ,but there is not universal-resolver-did-factom.json in dir /suites/implementations/. Is there any problem?

[Shigeya Suzuki]

I overlooked this issue, opened another one. This is the error caused by the missing file which @caict-develop-zhangbo mentioned. Adding the file will fix the above-mentioned exception.

@BernhardFuchs Please kindly add the file. thanks.

 ● Test suite failed to run

    Cannot find module '../implementations/universal-resolver-did-factom.json' from 'suites/did-resolution/default.js'

    Require stack:
      suites/did-resolution/default.js
      suites/did-resolution/did-resolution.spec.js

      11 |     require('../implementations/universal-resolver-did-ethr.json'),
      12 |     require('../implementations/universal-resolver-did-evan.json'),
    > 13 |     require('../implementations/universal-resolver-did-factom.json'),
         |     ^
      14 |     require('../implementations/universal-resolver-did-gatc.json'),
      15 |     require('../implementations/universal-resolver-did-github.json'),
      16 |     require('../implementations/universal-resolver-did-hcr.json'),

      at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
      at Object.<anonymous> (suites/did-resolution/default.js:13:5)

[Bernhard Fuchs]

@shigeya @caict-develop-zhangbo The issue created by shigeya #111 is the correct one to discuss the missing did:factom file.
This issue here, is about the testserver which can be run to host the testsuite. Will continue with that implementation, once the problems with the manually submitted files are solved.

[Bernhard Fuchs]

We have deployed a public instance of the did-test-suite to https://did-test-suite.uniresolver.io which is already used in the CI pipeline of the Universal Resolver. See decentralized-identity/universal-resolver#203

The original problem of this issue was solved by using the correct body for did-resolution:

[
  {
    "name": "did-resolution",
    "resolvers": [
      // implementations go here in the same format as the static files in the implementations folder
    ]
  }
]