Expand the list of supported Hadoop versions

Question

Expand the list of supported Hadoop versions

apennebaker opened this issue 11 years ago · comments

Could we adjust the build to be more flexible with respect to Hadoop versions? I'm not sure how users are expected to get Hadoop setup for the book:

On Mac, Homebrew installs Hadoop v1.2.1.
In Ubuntu, the online tutorials tend to specify Hadoop v1.0.3.
In Windows, tutorials tend to use v1.0.0.

None of these match the book code's demand of Hadoop v1.1.1.

/not_sure_if_book_code_is_general_enough_to_work_with_different_minor_versions

https://maven.apache.org/enforcer/enforcer-rules/versionRanges.html

Tom White · Answer 1 · Fri Jan 10 2014 06:30:12 GMT+0800 (China Standard Time)

I just pushed a change to fix this. E.g.

mvn package -Dhadoop.version=1.2.1
or
mvn package -Dhadoop.version=1.0.3

Sorry it took me so long - I didn't see the github issues until today.

apennebaker · Answer 2 · Sat Jan 11 2014 00:03:42 GMT+0800 (China Standard Time)

@tomwhite No worries! Thanks for addressing this. :)

Tom White · Answer 3 · Sat Jan 11 2014 05:48:53 GMT+0800 (China Standard Time)

I made another modification so you can specify both the distro and the precise version (or neither). See 9ffc052

I also ran the unit and integration tests against a bunch of different versions as follows:

HADOOP_HOME=/dev/hadoop-1.2.1 mvn clean verify -Dhadoop.version=1.2.1
HADOOP_HOME=/dev/hadoop-0.23.10 mvn clean verify -Dhadoop.distro=apache-23 -Dhadoop.version=0.23.10
HADOOP_HOME=/dev/hadoop-2.2.0 mvn clean verify -Dhadoop.distro=apache-2 -Dhadoop.version=2.2.0
HADOOP_HOME=/dev/hadoop-0.20.2-cdh3u5 mvn clean verify -Dhadoop.distro=cdh3 -Dhadoop.version=0.20.2-cdh3u5
HADOOP_HOME=~/dev/hadoop-2.0.0-cdh4.5.0 mvn clean verify -Dhadoop.distro=cdh4 -Dhadoop.version=2.0.0-cdh4.5.0