rhadoop giving null key value output

Question

rhadoop giving null key value output

Surender1984 opened this issue 7 years ago · comments

Surender1984 commented 7 years ago

I have integrated R with Hadoop using RHadoop and running below program from R.

library(rmr2)
library(rhdfs)
hdfs.init()
ints = to.dfs(1:100)
calc = mapreduce(input = ints, map = function(k, v) cbind(v, 2*v))
from.dfs(calc)

Map reduce job runs successfully but it's giving null output as below.

$key
NULL

$val
NULL

However if I use the option rmr.options(backend="local") then it's giving proper output.

It will be a great help if you can please help me on this and provide the solution.

huan liu · Answer 1 · Sun May 17 2020 16:20:16 GMT+0800 (China Standard Time)

I encountered the same problem. what should I do?

huan liu · Answer 2 · Sun May 17 2020 22:21:16 GMT+0800 (China Standard Time)

First, install rhadoop by install.packages () ; encounter the above problem.
Later, manually download rhadoop packages and their dependent packages (* tar.gz), using the method: R CMD INSTALL * .tar.gz, the above problem is solved.