recursive types + semi-auto derivation + federation does not work

[Sergey Torgashov]

Scala: v2.13.12
Caliban: v2.3.1

Consider a simple schema with recursive types:

import caliban.schema._
import zio.query._

case class IdArg(Id: Int)

object IdArg {
  implicit val idArgBuilder: ArgBuilder[IdArg] = ArgBuilder.gen
  implicit val idSchema: Schema[Any, IdArg] = Schema.gen
}

case class User(id: Int, group: UQuery[Role])
case class Role(d: Int, users: UQuery[List[User]])
case class Queries(users: UQuery[List[User]], roles: UQuery[List[Role]])

object Queries {
  // We are not going to make any queries, just to show schema derivation.
  def dummy: Queries = Queries(users = ZQuery.succeed(Nil), roles = ZQuery.succeed(Nil))
}

If we are going to make use of semi-auto derivation, here is a minimal example:

import caliban._
import caliban.schema._
import zio._

object ExampleApp extends ZIOAppDefault {
  // note that `userSchema` must be `def` to make the example working
  implicit def userSchema: Schema[Any, User] = Schema.gen
  implicit val roleSchema: Schema[Any, Role] = Schema.gen
  implicit val queriesSchema: Schema[Any, Queries] = Schema.gen

  val api = graphQL(RootResolver(Queries.dummy))

  override def run: Task[Unit] = Console.printLine(api.render)
}

Everything works at this point. However if we simply add federation with resolvers for both User and Role, then the example compiles but does not work anymore:

import caliban._
import caliban.federation._
import caliban.federation.v2_3._
import caliban.schema._
import zio._
import zio.query._

object ExampleAll extends ZIOAppDefault {
  implicit def userSchema: Schema[Any, User] = Schema.gen
  implicit val roleSchema: Schema[Any, Role] = Schema.gen
  implicit val queriesSchema: Schema[Any, Queries] = Schema.gen

  val api =
    graphQL(RootResolver(Queries.dummy)) @@ federated(
      EntityResolver[Any, IdArg, User] { _ => ZQuery.none },
      EntityResolver[Any, IdArg, Role] { _ => ZQuery.none }
    )

  override def run: Task[Unit] = Console.printLine(api.render)
}

An attempt to run it will fail with the exception:

Exception in thread "sbt-bg-threads-42" java.lang.ExceptionInInitializerError
at satorg.ExampleApp.main(ExampleApp.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at sbt.Run.invokeMain(Run.scala:144)
at sbt.Run.execute$1(Run.scala:94)
at sbt.Run.$anonfun$runWithLoader$5(Run.scala:121)
at sbt.Run$.executeSuccess(Run.scala:187)
at sbt.Run.runWithLoader(Run.scala:121)
at sbt.Defaults$.$anonfun$bgRunMainTask$7(Defaults.scala:1956)
at sbt.Defaults$.$anonfun$termWrapper$2(Defaults.scala:1927)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.util.Try$.apply(Try.scala:213)
at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:367)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)

Caused by: java.lang.NullPointerException: Cannot invoke "caliban.schema.Schema.optional()" because "this.ev$8" is null
at caliban.schema.GenericSchema$$anon$16.optional(Schema.scala:475)
at caliban.schema.CommonSchemaDerivation$$anon$1.$anonfun$toType$9(SchemaDerivation.scala:76)
at caliban.schema.Types$.$anonfun$collectTypes$20(Types.scala:149)
at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183)
at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179)
at scala.collection.immutable.List.foldLeft(List.scala:79)
at caliban.schema.Types$.collectTypes(Types.scala:148)
at caliban.schema.Types$.$anonfun$collectTypes$22(Types.scala:150)
at scala.Option.fold(Option.scala:263)
at caliban.schema.Types$.$anonfun$collectTypes$20(Types.scala:150)
at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183)
at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179)
at scala.collection.immutable.List.foldLeft(List.scala:79)
at caliban.schema.Types$.collectTypes(Types.scala:148)
at caliban.federation.FederationSupport.$anonfun$federate$5(FederationSupport.scala:93)
at scala.collection.immutable.List.flatMap(List.scala:293)
at caliban.federation.FederationSupport.federate(FederationSupport.scala:93)
at caliban.federation.FederationSupport$$anon$3.apply(FederationSupport.scala:39)
at caliban.GraphQL.$at$at(GraphQL.scala:164)
at caliban.GraphQL.$at$at$(GraphQL.scala:163)
at caliban.package$$anon$1.$at$at(package.scala:28)
at satorg.probes.caliban.fedrec.FederatedRecursiveApp1$.<clinit>(FederatedRecursiveApp1.scala:18)
... 18 more

[Sergey Torgashov]

I am pretty sure that a culprit is not the federation per se. I guess it is just using recursive types in different derivations simultaneously (RootResolver + EntityResolver). However, it is just easier to show the issue with the federation.

[Pierre Ricadat]

That's a known issue with Magnolia on Scala 2. Workaround: https://ghostdogpr.github.io/caliban/docs/schema.html#building-schemas-by-hand (note that you don't need to do that for all types, only some to break the recursion loop)

[Sergey Torgashov]

Thanks for the hint! Although it would be a bit painful in my case, but I'll give it a try.

I wonder though

or you will end up with a nasty runtime error from a NullPointerException.

how come it may end up with NPE in runtime in the first place?

I mean, it would be really more helpful if we always get compiler errors when Magnolia cannot handle something.
Otherwise such NPEs seem compromising the idea of compile-time type-safe code generation quite a bit.

[Pierre Ricadat]

I agree with you, it sucks. But it's out of our power (I opened an issue in magnolia at the time but it never really got resolved). I think it works better with Scala 3 (we don't use magnolia but native typeclass derivation).