Convert `Map[String, Any]` to case class

[samthebest]

Hi Miles,

For a while now I've wanted a neat way to convert a potentially nested Map[String, Any] to a case class, where each String key in the map will be the field name of the case class and some of the Any values could themselves be Map[String, Any] and thus represent nested case classes. The motivation is that I'm building a validation layer for Spark and the input data can be of any format.

The Any here is to be treated recursively as follows:

1. Recursive case: Any type that is a case class (so roughly speaking <: Product : TypeTag) should be converted to the corresponding case class (and the fields are recursed through).
2. Recursive case: Any type that is a native data structure is recursed through, e.g. List, Option and (nestings of such e.g. List[Option[...)
3. Base case: Anything that is not either of the above is left as is.

I've had some help from people on SO https://stackoverflow.com/q/55042252/1586965 in trying to use shapeless, and we have something kind of working for our case classes only consist of case 1 & 3. It seems the code we have come up with doesn't really handle case 2. We managed to modify it to handle Option and List, but it doesn't handle it when these are nested (i.e. Option[List[...).

For full code please see

https://github.com/samthebest/dump/blob/master/scala-utils/spark-validation/CaseClassFromMap.scala

and tests/examples including the case(s) we cannot make work.

https://github.com/samthebest/dump/blob/master/scala-utils/spark-validation/CaseClassFromMapTest.scala

Many thanks,

Sam

[Miles Sabin]

I recommend taking this to the shapeless gitter channel. My guess is that if the problem is in the nested case then you probably need to add a few more Lazy's.

[Georgi Krastev]

I think something like this could live in another library.
For more info on how to work with shapeless to achieve that I'd recommend "The Type Astronaut’s Guide to Shapeless" and for more concrete questions the Gitter channel linked above.