$ref to other files

[Guillaume]

Hello !

First thank you for this project and fastify-openapi-glue, it really helps for implementing a documentation-first API 🙂

I'm currently trying to use fastify-openapi-glue with an OpenAPI definition splitted in multiple files. From what I saw, it raises an error in this package.

If I understand correctly, this package doesn't retrieve the other files and thus I should provide a self-contained definition file ?
Or did I miss something ? 🙂

Just to give all the infos, I'm using the syntax for getting other files like that:

# OpenAPI file
# [...]
paths:
  /todos:
    $ref: "./list_todos.yaml"

And it fails in https://github.com/seriousme/openapi-schema-validator/blob/master/resolve.js#L33 as there is no # character in the uri, which gives the error Cannot read property 'length' of undefined.

Thanks in advance for the input

[Hans Klunder]

Hi,

thanks for the question !

As the Readme mentions:
"External references are not automatically resolved so you need to inline them yourself if required. "

So for now this is the only alternative. I will see if I can improve this, and at least improve the error message ;-)

Thanks for reporting.

Kind regards,
Hans

[Hans Klunder]

A fix to the error handling has been fixed in version 1.5.1

Kind regards,
Hans

[Guillaume]

My bad, missed that part in the readme, thanks for the info and the clearer error message :)

Kinds regards

[Hans Klunder]

Guillaume,

I have been thinking a bit and it was not too hard to add some functionality to aid in use cases like yours.
In the addSpecRef branch I have added the new functionality and I am curious if this solution would meet your use case.
(and whether the API naming feels natural and the Readme is clear enough ;-))

Once this is in I can add a parameter to fastify-openapi-glue to pass these kind of spec refs to openapi-schema-validator.

What do you think ?

Kind regards,
Hans

[Guillaume]

Hans,

I've looked a bit at the branch, it looks good and it can definitely help 😉

I have a few questions/suggestions. I'm aware that it's a small update to add some functionality to aid my use case, and therefore, it's of course not a big new feature. But I'm not sure of all the implications of this small change, so I prefer to ask:

Is the # still mandatory in the $ref value ?
In my case, in the URIs, there is often no "JSON pointer", so the whole sub-spec should be used. I haven't looked yet in the lib how it works, so maybe my question is stupid.

However, seeing that the sub-specs have to be explicitly specified, I was wondering:
I guess, ideally, the lib could infer and pull the sub-specs directly (based on the possibilities specified in the Openapi specs). For example, in my case, I have like 20-30 sub-specs, so that would make a lot of calls to addSpecRef. On the other hand, having a lot of files to load from disk (or worse, a lot of files to be pulled from the web) takes some time, so note that the argument of having one "self-contained spec" still somehow holds, as it doesn't worsen too much the start time of an app.

What do you think about this ? I know this is a lot more work, and I don't expect you to do it at all. I'm actually interested to know your opinion on the matter.
In fact, if you need help on that, I can try to do a PR 😉
In all cases, thanks for the input and the current improvements, it's already a big timesaver !

Kind regards,
Guillaume

[Hans Klunder]

Guillaume,

certainly no stupid question!
I added a test and it failed, so I fixed it :-)

As to the whole fetching thing I think the swagger crew went a bit overboard. For demo purposes it works great to do live lookups, but for production workloads you will want all your assets known (and tested ;-)) upfront and not depending on availability of third party sources.

Hence why nodejs downloads the packages explictly and also explictly names them in the package.json.
Deno solves this by caching sources locally, but while playing with Deno I quickly found my self importing third party packackes and redistributing from a central import ;-). Else you need to update each and every source file that references things like assert when a new version becomes available.

From an openapi-schema-validator architecture perspective I would like to keep things simple to minimize dependencies and complexity. Once I start adding a Deno style cache and various ways of retrieving data from a URL I will be taking on quite some work (as you mentioned ;-)) and dependencies :-(.

So if people want to go overboard and fetch data from git://, ftps:// or other URL's then I think they can still do so themselves and use addSpecRef to add the results without this module getting too much bloated ;-)

On the other hand: if you are afraid that startup time will be negatively impacted from opening many local files (which I doubt looking at how many files are present in a typical NPM package ;-)) you can make a script that does the preprocessing once, pull the inlined specfication and feed that to fastify-openapi-glue.

Thanks again for the feedback !

Kind regards,
Hans

[Guillaume]

Hans,

Thanks ! I'll try your branch and use addSpecRef in my project today or tomorrow 😉 (was planning do to it anyway)

Ahah you're right, I went indeed a little overboard about the load time of fetching all the local files, as you say the NPM ecosystem is already infamous for doing this and it isn't an issue.

I totally agree about the whole "fetching data/specs from external/third party sources doesn't really bring confidence". I wouldn't have a production workload do this kind of critical lookups either. I meant it more like "being full compatible with swagger/openapi specs", but this is indeed a good reason to not include this. Moreover, as you said, addSpecRef covers everything if some preprocessing is done in order to fetch the sub-specs 🙂

Thanks for the answer !

Kinds regards,
Guillaume

[Hans Klunder]

Btw: AJV uses the same strategy as I do: https://ajv.js.org/faq.html#generating-schemas-with-resolved-references-ref
Adding schemas is possible by using https://ajv.js.org/api.html#ajv-addschema-schema-object-object-key-string-ajv

Based on this I have added the option to add a $id attribute in the subspec and reversed the parameters of addSpecRef.

Kind regards,
Hans

[Hans Klunder]

Published as 1.6.0 on NPM

Thanks again !
Hans