Giters

circe / circe

Yet another JSON library for Scala

Home Page:https://circe.github.io/circe/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

a problem with current derivations

yurique opened this issue · comments

commented

Currently, we have the derivations defined in objects other than the type classes we actually want to derive:

  • Encoder.AsObject.derived
  • ConfiguredEncoder.AsObject.derived
  • Codec.AsObject.derived
  • ConfiguredCodec.AsObject.derived
  • etc

This works most of the time, but there is a problem with it. For example, the simplest encoder derivation looks like this:

case class Inner[A](
  field: A
) derives Encoder.AsObject

The compiler expands this into the following given:

object Inner {
  given [A: Encoder.AsObject]: Encoder.AsObject[Inner[A]] = Encoder.AsObject.derived
}

Notice how there is a Encoder.AsObject (not Encoder) context bound for the A type parameter. This is the root of the problem.

Say we add another case class:

case class Outer(
  a: Option[Inner[String]],
) derives Encoder.AsObject

which gets expanded:

object Outer {
  given Encoder.AsObject[Outer] = Encoder.AsObject.derived
}

Now, when the derivation for Outer kicks in, it will try to find a given Encoder[A] (not Encoder.AsObject[A]) (and this is what we actually want here) for all the fields in the case class.
We have one field, a: Option[Inner[String]], and in order to "find" the encoder for the Option the compiler needs to find a given encoder for Inner[String] first. We would expect it to find and use the derived given for the Inner[A], but it will not exist for Inner[String] since there is no Encoder.AsObject[String] (which is expected, there is only a non-.AsObject encoder for Strings). So at this point, our derivation sees that there is no encoder for Option[Inner[String]] and moves on to derive it treating Option as a regular product type.

And we end up with this encoding:

  Outer(
    a = Some(Inner("c"))
  ).asJson.spaces2 
{
  "a" : {
    "Some" : {
      "value" : {
        "field" : "c"
      }
    }
  }
}

which is obviously incorrect.

To work this around, we can define the encoder for the Inner[A} manually:

object Inner {
  given [A: Encoder /* .AsObject - not AsObject! */]: Encoder.AsObject[Inner[A]] = Encoder.AsObject.derived
}

and with this, the derivation will work correctly.

But this means we cannot reliably define codecs using derives ....

In order to fix it, the derivation has to be happening "inside" the Encoder, not in Encoder.AsObject.

It would look like this:

case class Inner[A](
  field: A
) derives Encoder

which will get expanded into this given:

object Inner {
  given [A: Encoder]: Encoder[Inner[A]] = Encoder.AsObject.derived
}

And everything will work as expected.

Now, the problem doesn't end here: we also have ConfiguredEncoder.derived, as well as Codec.derived and ConfiguredCodec.derived, which have the same problems.

I think the solution would be to have just a single point of derivation, Encoder.derived (and Decoder.derived). We would have to drop the Codec.derived as well as ConfiguredEncoder.derived and ConfiguredCodec.derived.

Dropping Configured* means we would have to add the using Configuration to the Encoder.derived (with a default Configuration.default).

object Codec extends ... with CodecDerivation ...

private[circe] trait EncoderDerivation:
  inline final def derived[A](using
    inline A: Mirror.Of[A],
    inline configuration: Configuration = Configuration.default
  ): Encoder.AsObject[A] = ...

But this seems to be a very big and breaking change 🤔.

If we keep the current separation, maybe there is a way to make it respect the existing givens, I got referenced to how it is done in kittens (here scala/scala3#19391 (comment)):
https://github.com/typelevel/kittens/blob/master/core/src/main/scala-3/cats/derived/Derived.scala - but have zero understanding of how it works 😅

commented

If we keep the current separation, maybe there is a way to make it respect the existing givens, I got referenced to how it is done in kittens (here scala/scala3#19391 (comment)):
https://github.com/typelevel/kittens/blob/master/core/src/main/scala-3/cats/derived/Derived.scala - but have zero understanding of how it works 😅

It basically abstracts this pattern with summonFrom and fallback to deriving into an opaque type.

commented

Apart from the issues you mentioned, I see that derived is an inline method which creates an anonymous class.
This is an anti-pattern because it will result in a new class file per derived instance.

commented

This is an anti-pattern because it will result in a new class file per derived instance.

Is there a better way?

Since we are basically generating instances of Encoder / Decoder traits, some custom code has to be "generated" somewhere, one way or another. At the very least it will be a lambda, but it will also be an anonymous class (which extends FuntionX[...])

commented

Is there a better way?

I'm preparing a change

commented

This is awesome! :)