Remote node launching by machine tag
vifr9883 opened this issue · comments
Hello,
I have 2 computers in a local network (192.168.1.0/24) setup as follows:
- Master (192.168.1.203)
ROS_MASTER_URI= http://192.168.1.203:11311
ROS_IP= 192.168.1.203 - Slave (192.168.1.211)
ROS_MASTER_URI= http://192.168.1.203:11311
ROS_IP= 192.168.1.211
I am trying to launch a node in Slave from Master using remote.launch file in Master with the machine tag. The file looks as follows:
<launch>
<machine name="Slave"
address="192.168.1.211"
default="false"
env-loader="/opt/ros/noetic/env.sh"
user="user"
password="password"
/>
<node pkg="my_pkg" type="my_node.py" name="my_node" machine="Slave"/>
</launch>
The node my_node.py can run in Slave without issues and the connection works well when I launch the node manually through SSH. However, if I run the following line in Master:
roslaunch my_remote_pkg remote.launch
I receive the following output:
... logging to /redacted_path.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
started roslaunch server http://192.168.1.203:34417/
remote[192.168.1.211-0] starting roslaunch
remote[192.168.1.211-0]: creating ssh connection to 192.168.1.211:22, user[user]
launching remote roslaunch child with command: [env ROS_MASTER_URI=http://192.168.1.203:11311 /opt/ros/noetic/env.sh roslaunch -c 192.168.1.211-0 -u http://192.168.1.203:34417/ --run_id 85500600-6e77-11e
c-b7f9-3b57dbe79019 --sigint-timeout 15.0 --sigterm-timeout 2.0]
remote[192.168.1.211-0]: ssh connection created
SUMMARY
========
CLEAR PARAMETERS
* /redacted
PARAMETERS
* /redacted
MACHINES
* Slave
NODES
/
my_node (my_pkg/my_node.py)
auto-starting new master
process[master]: started with pid [38837]
ROS_MASTER_URI=http://192.168.1.203:11311
setting /run_id to cf5851d4-6e78-11ec-b7f9-3b57dbe79019
process[rosout-1]: started with pid [38866]
started core service [/rosout]
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/roslaunch/remote.py", line 208, in launch_remote_nodes
code, msg, val = api.launch(xml)
File "/usr/lib/python3.8/xmlrpc/client.py", line 1109, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python3.8/xmlrpc/client.py", line 1450, in __request
response = self.__transport.request(
File "/usr/lib/python3.8/xmlrpc/client.py", line 1153, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python3.8/xmlrpc/client.py", line 1165, in single_request
http_conn = self.send_request(host, handler, request_body, verbose)
File "/usr/lib/python3.8/xmlrpc/client.py", line 1278, in send_request
self.send_content(connection, request_body)
File "/usr/lib/python3.8/xmlrpc/client.py", line 1308, in send_content
connection.endheaders(request_body)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 922, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 787, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/roslaunch/__init__.py", line 347, in main
p.start()
File "/opt/ros/noetic/lib/python3/dist-packages/roslaunch/parent.py", line 316, in start
self.runner.launch()
File "/opt/ros/noetic/lib/python3/dist-packages/roslaunch/launch.py", line 677, in launch
succeeded, failed = self._launch_nodes()
File "/opt/ros/noetic/lib/python3/dist-packages/roslaunch/launch.py", line 391, in _launch_nodes
r_succ, r_fail = self.remote_runner.launch_remote_nodes()
File "/opt/ros/noetic/lib/python3/dist-packages/roslaunch/remote.py", line 217, in launch_remote_nodes
errno, msg = e
TypeError: cannot unpack non-iterable gaierror object
[192.168.1.211-0] killing on exit
remote[192.168.1.211-0]: unable to contact [192.168.1.211] to shutdown remote processes!
[rosout-1] killing on exit
[master] killing on exit
Additionally, here is part of redacted_path.log:
...
[roslaunch][INFO] 2022-01-05 16:43:28,813: process[rosout-1]: started with pid [38946]
[roslaunch][INFO] 2022-01-05 16:43:28,814: ... successfully launched [rosout-1]
[roslaunch][INFO] 2022-01-05 16:43:28,814: load_parameters starting ...
[roslaunch][INFO] 2022-01-05 16:43:28,817: ... load_parameters complete
[roslaunch][INFO] 2022-01-05 16:43:28,817: launch_nodes: launching local nodes ...
[roslaunch][INFO] 2022-01-05 16:43:28,817: launch_nodes: launching remote nodes ...
[roslaunch.pmon][INFO] 2022-01-05 16:43:28,819: ProcessMonitor.shutdown <ProcessMonitor(ProcessMonitor-1, started daemon 140090944329472)>
[roslaunch.pmon][INFO] 2022-01-05 16:43:28,832: ProcessMonitor._post_run <ProcessMonitor(ProcessMonitor-1, started daemon 140090944329472)>
[roslaunch.pmon][INFO] 2022-01-05 16:43:28,833: ProcessMonitor._post_run <ProcessMonitor(ProcessMonitor-1, started daemon 140090944329472)>: remaining procs are [<roslaunch.remoteprocess.SSHChildROSLaunchProcess object at 0x7f6a0b344160>, <roslaunch.nodeprocess.LocalProcess object at 0x7f6a07a4cd60>, <roslaunch.nodeprocess.LocalProcess object at 0x7f6a07a3d820>]
[roslaunch.pmon][INFO] 2022-01-05 16:43:28,833: ProcessMonitor exit: killing 192.168.1.211-0
[roslaunch][INFO] 2022-01-05 16:43:28,834: [192.168.1.211-0] killing on exit
[roslaunch][ERROR] 2022-01-05 16:43:28,834: remote[192.168.1.211-0]: unable to contact [192.168.1.211] to shutdown remote processes!
[roslaunch.remoteprocess][INFO] 2022-01-05 16:43:28,835: remote[192.168.1.211-0]: closing ssh connection
[roslaunch.remoteprocess][INFO] 2022-01-05 16:43:28,836: remote[192.168.1.211-0]: ssh connection closed
[roslaunch.pmon][INFO] 2022-01-05 16:43:28,836: ProcessMonitor exit: killing rosout-1
[roslaunch][INFO] 2022-01-05 16:43:28,836: [rosout-1] killing on exit
[roslaunch][INFO] 2022-01-05 16:43:28,837: process[rosout-1]: killing os process with pid[38946] pgid[38946]
[roslaunch][INFO] 2022-01-05 16:43:28,837: [rosout-1] sending SIGINT to pgid [38946]
[roslaunch][INFO] 2022-01-05 16:43:28,837: [rosout-1] sent SIGINT to pgid [38946]
[roslaunch][INFO] 2022-01-05 16:43:29,038: process[rosout-1]: SIGINT killed with return value 0
[roslaunch.pmon][INFO] 2022-01-05 16:43:29,038: ProcessMonitor exit: killing master
[roslaunch][INFO] 2022-01-05 16:43:29,038: [master] killing on exit
[roslaunch][INFO] 2022-01-05 16:43:29,038: process[master]: killing os process with pid[38917] pgid[38917]
[roslaunch][INFO] 2022-01-05 16:43:29,038: [master] sending SIGINT to pgid [38917]
[roslaunch][INFO] 2022-01-05 16:43:29,038: [master] sent SIGINT to pgid [38917]
I am wondering if this issue is due to my configuration or something else. Again, everything works fine if I manually start my_node in Slave through SSH, so this is not high priority.
Thank you for your time.
@vifremel Hey did you figure it out? I am facing the exact same issue. I have been struggling with this for a long time now.
I never figured this out. It was not critical for my application so I ended up just launching the nodes after SSH manually. Sorry.
@vifremel I think you should include both your local and remote host names in the /opt/ros/noetic/env.sh.
Unfortunately, I no longer have the hardware/resources to test this. I will close this issue assuming the proposed solution by @mohsafwat23 works. Thank you!