[ERROR] Some problems were encountered while processing the POMs
DachuanZhao opened this issue · comments
When I run mvn -Pscala-2.11.12 clean install -Dspark.version=2.4.4 -Dtensorflow.hadoop.version=2.8.5 , It raises error .
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[WARNING] 'artifactId' contains an expression but should be a constant. @ line 7, column 17
[ERROR] 'artifactId' with value 'spark-tfrecord_${scala.binary.version}' does not match a valid id pattern. @ line 7, column 17
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-core_${scala.binary.version}:jar with value 'spark-core_${scala.binary.version}' does not match a valid id pattern. @ line 379, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-sql_${scala.binary.version}:jar with value 'spark-sql_${scala.binary.version}' does not match a valid id pattern. @ line 385, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-mllib_${scala.binary.version}:jar with value 'spark-mllib_${scala.binary.version}' does not match a valid id pattern. @ line 391, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-mllib_${scala.binary.version}:test-jar with value 'spark-mllib_${scala.binary.version}' does not match a valid id pattern. @ line 397, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.scalatest:scalatest_${scala.binary.version}:jar with value 'scalatest_${scala.binary.version}' does not match a valid id pattern. @ line 289, column 33
[ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for org.scalatest:scalatest_${scala.binary.version}:jar with value 'scalatest_${scala.binary.version}' does not match a valid id pattern. @ line 280, column 37
@
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project com.linkedin.sparktfrecord:spark-tfrecord_${scala.binary.version}:0.3.4 (/Users/zhaodachuan/test/spark-tfrecord/pom.xml) has 7 errors
[ERROR] 'artifactId' with value 'spark-tfrecord_${scala.binary.version}' does not match a valid id pattern. @ line 7, column 17
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-core_${scala.binary.version}:jar with value 'spark-core_${scala.binary.version}' does not match a valid id pattern. @ line 379, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-sql_${scala.binary.version}:jar with value 'spark-sql_${scala.binary.version}' does not match a valid id pattern. @ line 385, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-mllib_${scala.binary.version}:jar with value 'spark-mllib_${scala.binary.version}' does not match a valid id pattern. @ line 391, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-mllib_${scala.binary.version}:test-jar with value 'spark-mllib_${scala.binary.version}' does not match a valid id pattern. @ line 397, column 25
[ERROR] 'dependencies.dependency.artifactId' for org.scalatest:scalatest_${scala.binary.version}:jar with value 'scalatest_${scala.binary.version}' does not match a valid id pattern. @ line 289, column 33
[ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for org.scalatest:scalatest_${scala.binary.version}:jar with value 'scalatest_${scala.binary.version}' does not match a valid id pattern. @ line 280, column 37
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
``Scala 2.11 is no longer supported on master.
To build Scala 2.11 for Spark 2.4, please check out branch `spark-2.4`. Note we don't update branch `spark-2.3` and `spark-2.4` anymore.
With your command.
mvn -Pscala-2.11.12 clean install -Dspark.version=2.4.4 -Dtensorflow.hadoop.version=2.8.5
you will get the following error:
[ERROR] Failed to execute goal on project spark-tfrecord_2.11: Could not resolve dependencies for project com.linkedin.sparktfrecord:spark-tfrecord_2.11:jar:0.2.6: Could not find artifact org.tensorflow:tensorflow-hadoop:jar:2.8.5 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
I don't see tensorflow.hadoop 2.8.5, maybe it's your local build?
https://search.maven.org/search?q=tensorflow-hadoop
mvn -Pscala-2.11.12 clean install -Dspark.version=2.4.4 -Dtensorflow.hadoop.version=
Thanks for your reply . Finally , I just download spark-tensorflow-connector_2.11-1.14.0.jar package from https://mvnrepository.com/artifact/org.tensorflow/spark-tensorflow-connector .
No problem. I thought you need to build some special version.
If you can use built binary, Spark-tfrecord 0.2.6 may work for you too.
https://mvnrepository.com/artifact/com.linkedin.sparktfrecord/spark-tfrecord