vangj / vagrant-hadoop-2.4.1-spark-1.0.1

Vagrant project to spin up a cluster virtual machines with Hadoop v2.4.1 and Spark v1.0.1

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

problem with curl command when install hadoop from remote file

casertap opened this issue · comments

Hi,
I would like to try this vm so I run the vagrantfile using
vagrant up --provider virtualbox

and I get this output showing the problem with the curl command

==> node-4: setup java                                                                                                                                                [0/133]
==> node-4: install open jdk
==> node-4: Loaded plugins: fastestmirror
==> node-4: Determining fastest mirrors
==> node-4:  * base: mirror.prolocation.net
==> node-4:  * extras: mirror.prolocation.net
==> node-4:  * updates: mirror.widexs.nl
==> node-4: Setting up Install Process
==> node-4: No package jdk-8u25-linux-i586 available.
==> node-4: Error: Nothing to do
==> node-4: setting up java
==> node-4: creating java environment variables
==> node-4: Running provisioner: shell...
    node-4: Running: /var/folders/wl/nfnck8ld04n8hr183hm501r80000gn/T/vagrant-shell20150827-1811-1b7ruv5.sh
==> node-4: setup hadoop
==> node-4: install hadoop from remote file
==> node-4: c
==> node-4: u
==> node-4: r
==> node-4: l
==> node-4: :
==> node-4:  
==> node-4: (
==> node-4: 6
==> node-4: )
==> node-4:  Couldn't resolve host '..'
==> node-4: tar (child): 
==> node-4: /vagrant/resources/hadoop-2.6.0.tar.gz: Cannot open
==> node-4: : No such file or directory
==> node-4: tar (child): 
==> node-4: Error is not recoverable: exiting now
==> node-4: tar: Child returned status 2
==> node-4: tar: Error is not recoverable: exiting now
==> node-4: creating hadoop directories
==> node-4: copying over hadoop configuration files
==> node-4: cp: 
==> node-4: target `/usr/local/hadoop/etc/hadoop' is not a directory
==> node-4: creating hadoop environment variables
==> node-4: Running provisioner: shell...
    node-4: Running: /var/folders/wl/nfnck8ld04n8hr183hm501r80000gn/T/vagrant-shell20150827-1811-1lj24tu.sh
==> node-4: setup hadoop slaves
==> node-4: modifying /usr/local/hadoop/etc/hadoop/slaves
==> node-4: adding node3
==> node-4: adding node4
==> node-4: /tmp/vagrant-shell: line 22: /usr/local/hadoop/etc/hadoop/slaves: No such file or directory
==> node-4: /tmp/vagrant-shell: line 22: /usr/local/hadoop/etc/hadoop/slaves: No such file or directory
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

Sorry about the issue.
I was mistaking your repo with this one: https://github.com/dnafrance/vagrant-hadoop-spark-cluster
Your vagrantfile works until node2 where it shows:

==> node2: executing ssh-copy-id
==> node2: copy ssh key to node3
==> node2: /usr/bin/ssh-copy-id: line 41: sshpass: command not found
==> node2: copy ssh key to node4
==> node2: /usr/bin/ssh-copy-id: line 41: sshpass: command not found

I replace your method to install sshpass by this

function installSSHPass {
    #yum -y install sshpass
    yum -y install wget
    wget http://pkgs.repoforge.org/sshpass/sshpass-1.05-1.el6.rf.i686.rpm
    rpm -ivh sshpass-1.05-1.el6.rf.i686.rpm                                       
}                                      

To solve this issue