Cascading / CoPA

Cascading plus City of Palo Alto open data

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Gradle build can't find dependencies with Cascading 2.1.x

royseto opened this issue · comments

On my system, the latest build.gradle fails with any Cascading 2.1.x version because it can't find all the dependencies. When I revert cascadingVersion to any 2.0.x version, it works again.

See https://gist.github.com/4705282

Great to find any bugs in the build process. I have a hunch that we're seeing the same issue in both Java and Clojure.

We're working with JVM 1.6, and many people have found issues with 1.7 so that is most likely the root cause here. So the next step would be to try with JVM 1.6 and then both Cascading 2.1.* and Cascalog should build correctly.

Other potential versioning issues:

I'm using Gradle 1.3, but 1.4 should be working fine - will upgrade to verify. There was an issue in 1.1, resolved when they moved to 1.2, and we've been using 1.3+ due to support for Scala

Not sure from the logs about the Hadoop distro, but 1.x should be fine. We've seen problems in CDH4, since it's no longer byte-code compatible with Apache Hadoop, but that would be unlikely for this code.

I'll email a section from the book which gives a simple recipe for Leiningen, but from the logs you've already got that running fine. We have seen problems with Leiningen installs, but once you get to the point of a "lein uberjar" its running correctly.

Tested with Gradle 1.4, and that looks good.
I've updated the README to show versioning dependencies.

I switched from JDK 1.7 to JDK 1.6, and "gradle clean jar" followed by running the hadoop job now works out of the box.

"lein uberjar" still has the same problem as last night - I'll make a separate issue in github for that.

$ java -version
java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-11M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)