Macros: Size of generated code is exponential

[Florian Kirmaier]

The size of the code, generated by macros gets pretty big.
The generated code is exponential to the depth of the case-class-hierarchy.

I think the reason for this behavior is, that the picklers are generated both for serializing and deserializing.

In the following Sample, the pickler for TestCaseClass1 is created 16 times.

case class TestCaseClass5(x: TestCaseClass4)
case class TestCaseClass4(x: TestCaseClass3)
case class TestCaseClass3(x: TestCaseClass2)
case class TestCaseClass2(x: TestCaseClass1)
case class TestCaseClass1(x: Int)

[Otto Chrons]

This seems to be just one of many examples where automatic derivation of picklers is a bad practise. It's recommended to explicitly declare the implicit picklers in one place and use them instead of relying on automatic generation at every call site.

For example https://scalafiddle.io/sf/ZRO6far/0 "solves" this issue through explicit pickler definition.

Or am I missing something?

[Florian Kirmaier]

I understand that the automatic derivation is currently bad practice, for numerous reasons.
But the generated code shouldn't grow exponential.

But the automatic derivation also has some advantages. It makes experimenting much easier. It allowed me to switch from upickle to boopickle without many changes in my codebase.

I think, by fixing this issue, the generated code for other use cases might reduce also.

I think the problem happens in the following file:
https://github.com/suzaku-io/boopickle/blob/master/boopickle/shared/src/main/scala/boopickle/PicklerMaterializersImpl.scala

The pickler- and unpicklerlogic both instantiate an own pickler in their own code.
Moving it to a def, which is shared between both methods, might fix this issue.
But it would be important, to check whether it influences the performance of boopickle.

[Otto Chrons]

Yea, there could be some room for optimization there, which would cut down the number of derived picklers by half, but it wouldn't solve the root cause. It might make sense to collect the types of case class accessors separately, assign the derived picklers for each type into an object private val and then use those in both pickle and unpickle methods.

Want to experiment with it? :)

[Florian Kirmaier]

I don't have time for it at the moment, but maybe in the future.