standalone: cannot deploy topology
jsafrane opened this issue · comments
I am following standalone/README.md up to the command heketi-cli topology load --json=topology_libvirt.json:
[vagrant@storage0 ~]$ heketi-cli topology load --json=topology_libvirt.json
Creating cluster ... ID: 6ce349156b7351fb5617ce0427416069
Creating node 192.168.10.100 ... ID: ed64a4975c0abb720eb9c2cde4009074
Adding device /dev/vdb ... Unable to add device: /bin/bash: pvcreate: command not found
<snip>
Creating node 192.168.10.101 ... Unable to create node: /bin/bash: gluster: command not found
It turns out, that heketi connects via ssh back to storage0 and tries to execute pvcreate, which is not in $PATH. From some reason, bash in ssh does not have /sbin in $PATH and fails. The same happens with gluster binary - /usr/sbin is not in $PATH either.
In addition, if I fix $PATH, pvcreate fails to create anything as it is run by user vagrant and not root. I needed to change /etc/heketi.json to log in as root to remote machines and distribute correct public key to /root/.ssh/authorized_keys. With these changes, heketi-cli topology load works like charm.
O.o I need to fix this. Thanks!
I would like to have a go at fixing this bug. I have hit the same issue and exploring through the code debug it.
Luis,
Please wait for a day before you debug this.
@raghavendra-talur Please, go fo it! 👍
Root cause:
The $PATH available when ssh is used to login vs when ssh is used to execute a command directly are different. Here is the detailed output for the same.
Case 1: As user heketi, login to machine as vagrant and echo $PATH.
bash-4.2$ ssh -i /etc/heketi/insecure_private_key vagrant@192.168.10.100
Last login: Thu Sep 1 08:59:49 2016 from 192.168.121.1
[vagrant@storage0 ~]$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/vagrant/.local/bin:/home/vagrant/bin
Case 2: As user heketi, echo $PATH.
bash-4.2$ ssh -i /etc/heketi/insecure_private_key vagrant@192.168.10.100 'echo $PATH'
/usr/local/bin:/usr/bin
Where is pvcreate?
bash-4.2$ which pvcreate
/usr/sbin/pvcreate
This is the reason we can't find pvcreate, as it is in /usr/sbin.
The fix will be in heketi code. heketi/heketi#463