Serialization with Epic and Breeze Dependencies

[Evan Sparks]

My project depends on both epic and breeze (because they're both awesome!)

Unfortunately, in the process of upgrading to breeze 0.12 (we also depend on MLlib which depends on breeze 0.12 as of recent versions of spark) our code broke because the current English POS model:

"org.scalanlp" %% "epic-pos-en" % "2015.2.19"

Depends on breeze 0.11-M0.

If my project depends on both breeze 0.12 and this model, when I go to load the model I get the following:

scala> val model: SemiCRF[Any, String] = epic.models.NerSelector.loadNer("en").get
java.lang.ClassCastException: cannot assign instance of epic.lexicon.SimpleLexicon$SerializedForm to field epic.constraints.LabeledSpanConstraints$LayeredTagConstraintsFactory.lexicon of type epic.constraints.TagConstraints$Factory in instance of epic.constraints.LabeledSpanConstraints$LayeredTagConstraintsFactory
	at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2083)
	at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1995)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
	at epic.models.ClassPathModelLoader.load(ModelLoader.scala:23)
	at epic.models.DelegatingLoader.load(ModelLoader.scala:33)
	at epic.models.NerSelector$.loadNer(NerModelLoader.scala:12)
	at .<init>(<console>:8)
	at .<clinit>(<console>)
	at .<init>(<console>:7)
	at .<clinit>(<console>)
	at $print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:734)
	at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:983)
	at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:573)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:604)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:568)
	at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:760)
	at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:805)
	at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:717)
	at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:581)
	at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:588)
	at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:591)
	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:882)
	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:837)
	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:837)
	at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
	at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:837)
	at scala.tools.nsc.interpreter.ILoop.main(ILoop.scala:904)
	at xsbt.ConsoleInterface.run(ConsoleInterface.scala:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:101)
	at sbt.compiler.AnalyzingCompiler.console(AnalyzingCompiler.scala:76)
	at sbt.Console.sbt$Console$$console0$1(Console.scala:22)
	at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply$mcV$sp(Console.scala:23)
	at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply(Console.scala:23)
	at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply(Console.scala:23)
	at sbt.Logger$$anon$4.apply(Logger.scala:85)
	at sbt.TrapExit$App.run(TrapExit.scala:248)
	at java.lang.Thread.run(Thread.java:724)

I'm not sure what the best way forward is here. I can envision:

1. no longer relying on java serialization for pre-trained models
2. publishing a version of the models for several versions of breeze.
3. some classpath hacks (?)