linkedin / photon-ml

A scalable machine learning library on Apache Spark

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Inconsistency in OptimizerState between photon-api and photon-ml

maver1ck opened this issue · comments

OptimizerState class have different field in those modules.
When building shaded jar (with photon-all) this gives nice exceptions

Exception in thread "main" java.lang.NoSuchMethodError: com.linkedin.photon.ml.optimization.OptimizerState.loss()D
        at com.linkedin.photon.ml.optimization.Optimizer.setAbsTolerances(Optimizer.scala:69)
        at com.linkedin.photon.ml.optimization.Optimizer.optimize(Optimizer.scala:176)
        at com.linkedin.photon.ml.optimization.DistributedOptimizationProblem.run(DistributedOptimizationProblem.scala:116)
        at com.linkedin.photon.ml.optimization.DistributedOptimizationProblem.run(DistributedOptimizationProblem.scala:104)
        at com.linkedin.photon.ml.ModelTraining$$anonfun$6.apply(ModelTraining.scala:180)
        at com.linkedin.photon.ml.ModelTraining$$anonfun$6.apply(ModelTraining.scala:173)
        at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124)
        at scala.collection.immutable.List.foldLeft(List.scala:84)
        at com.linkedin.photon.ml.ModelTraining$.trainGeneralizedLinearModel(ModelTraining.scala:173)
        at com.linkedin.photon.ml.ModelTraining$.trainGeneralizedLinearModel(ModelTraining.scala:68)
        at com.linkedin.photon.ml.Driver.train(Driver.scala:345)
        at com.linkedin.photon.ml.Driver.run(Driver.scala:174)
        at com.linkedin.photon.ml.Driver$.main(Driver.scala:702)
        at com.linkedin.photon.ml.Driver.main(Driver.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:                                                                                                43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(                                                                                                SparkSubmit.scala:755)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Please merge PR.

Resolved by #291