Static Schema Index

[Jakub Kozłowski]

Currently, we have DynamicSchemaIndex which has roughly these functions:

• schemas: List[Schema[_]]
• services: List[Service[_]]
• metadata: Map[String, Document]

It's only useful in Dynamic, of course, which implies that you have a runtime instance of dynamic.Model, which is hard to construct if you don't have a JSON dump of a model, or you're on the JVM and have a smithy-model Model.

Looking up schemas at runtime could be useful in certain usecases, like if you have an idRef trait somewhere and want to do more with it. To me, the really useful bit would be looking up metadata: currently, any metadata you want to access at runtime in the "static" world of Smithy4s has to be copied into a trait (so that it's available as a hint).

A potential solution to this inconvenience would be to have some sort of "static schema index" type. Its interface could be well-defined in smithy4s (and quite similar to DSI, if not the same type), and we'd generate an object implementing it, somewhere - this could be like BuildInfo.

There are a couple problems with this:

• users would have to pass this (static) SchemaIndex (let's call it SSI from now on) to the interpreters somehow, and even if it's an implicit, it'd be an orphan instance - so an import would be necessary
• "staged" codegen: smithy4s allows generating some code in one module, then some more code in another module, then some more. In fact, these modules needn't be aware of each other, and could be pulled in by an application that doesn't even use codegen. How do we "gather" all the "sub-indexes"?
• (possibly simpler) where do we put this object? Especially if there's going to be more than one of these on the classpath.

Random ideas:

• assume we only support apps that do have the codegen plugin? (so that there's always "a place to gather things in and generate")
• store a list of shapes/services/metadata in the smithy4sGenerated.smithy file, and then render a file made from that?
• file size: a static schema index is likely going to be an enormous string, and we have to be mindful of the limitations of Scala, the JVM, as well as JS/native.

The more I write about it the more I think this is an insane and overcomplicated idea, but it's good to at least have some reasons written down for why it's going to be a problem.

[Olivier Mélois]

That's a big no for me, in that it's a fair bit of maintenance burden for a usecase that has yet to find proper motivation. The only point that I'd consider accepting at this time, in smithy4s is this :

store a list of shapes/services/metadata in the smithy4sGenerated.smithy file

I think that'd be acceptable : we could render aMap[ShapeId, FQN] in the metadata. This would empower you to develop a third party build plugin downstream to render your static schema indexes without requiring tribal knowledge of how things get rendered and what are the escaping rules. Later on, if/when you prove the relevance of the idea, we could revisit upstreaming it here.

[Jakub Kozłowski]

Replacing with #1436