tomwhite / hadoop-book

Example source code accompanying O'Reilly's "Hadoop: The Definitive Guide" by Tom White

Home Page:http://www.hadoopbook.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Chapter 2: MapReduce: hadoop command doesn't work without class package

Tradunsky opened this issue · comments

On the 25th page of the book there is a command to execute MaxTemperature map reduce job:

hadoop MaxTemperature input/ncdc/sample.txt output

with the execution logs output. However, as for my environment the execution result is different:

ERROR: MaxTemperature is not COMMAND nor fully qualified CLASSNAME.

After a quick look it turns out that it works only with class specified including package:

hadoop oldapi.MaxTemperature input/ncdc/sample.txt output

Not sure if it is something important, but want to highlight in the case somebody stumble as I did.

uname -a
Darwin viacheslavt-mac.local 17.6.0 Darwin Kernel Version 17.6.0: Tue May 8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64 x86_64
hadoop version
Hadoop 3.0.3

same error, thanks for reminding.

According to latest hadoop shellcommand documentation, "the class must be part of the package":
http://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-common/CommandsManual.html#CLASSNAME

Probably the error is happening only for newer versions of hadoop

What if I want to run it with the new api?

@dpstart I can run it with

hadoop jar hadoop-examples.jar MaxTemperature input/ncdc/sample.txt output

Confirm that the command by @gyosh works on new Hadoop version.

@dpstart I can run it with

hadoop jar hadoop-examples.jar MaxTemperature input/ncdc/sample.txt output

it works!

Same error, drove me crazy, chapter two and already struggling to follow the instructions. Thanks.
Both works for me, including the package as well as command by @gyosh