Issue: Bug/Performance Issue [Physical Robot] - Roslaunch not working

Question

Issue: Bug/Performance Issue [Physical Robot] - Roslaunch not working

rickstaa opened this issue 5 years ago · comments

Rick Staa commented 5 years ago

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04 singularity v3.2.1 container
Python version: 2.7.12 & 3.5.2
Installed using pip or ROS: ROS
Camera: Kinect v2 (Libfreenect2 library)
Gripper: Parallel jaw gripper
Robot: Emika Franka Panda robot
GPU model (if applicable): Nvidia M1000M GPU
ROS version: kinetic

Describe what you are trying to do

I am trying to test your GQCNN ROS package on an Emika Franka Panda robot.

Problem description

Before applying it on the real robot I first am testing out the GQCNN package inside a gazebo simulation. To do this I tried launching the ROS node both in a python2 and python3 environment.

Python 2.7.15 virtual environment (virtualenv package)

When I try to launch the ROS node using the roslaunch gqcnn grasp_planning_service.launch model_name:=GQCNN-4.0-PJ command inside the pyhton 2.7 enviroment I get the following error:

Traceback (most recent call last):
  File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 45, in <module>
    from autolab_core import YamlConfig
ImportError: No module named autolab_core

Due to this error I assumed I needed to install the packages present inside the setup.py file. I, therefore, created a python 2.7.15 virtual environment, sourced the environment and executed the following commands:

python -m pip install -r gqcnn/requirements/p2/gpu_requirements.txt

However, if while in the enviroment if I want to build the ROS package again I get the following error:

ImportError: "from catkin_pkg.package import parse_package" failed: No module named catkin_pkg.package
Make sure that you have installed "catkin_pkg", it is up to date and on the PYTHONPATH.
CMake Error at /opt/ros/kinetic/share/catkin/cmake/safe_execute_process.cmake:11 (message):
  execute_process(/virtualenvs/test/bin/python
  "/opt/ros/kinetic/share/catkin/cmake/parse_package_xml.py"
  "/opt/ros/kinetic/share/catkin/cmake/../package.xml"
  "/catkin_ws/build/catkin/catkin_generated/version/package.cmake") returned
  error code 1
Call Stack (most recent call first):
  /opt/ros/kinetic/share/catkin/cmake/catkin_package_xml.cmake:74 (safe_execute_process)
  /opt/ros/kinetic/share/catkin/cmake/all.cmake:167 (_catkin_package_xml)
  /opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:20 (include)
  CMakeLists.txt:56 (find_package)

This is is not strange since the rospackages are installed on the global python 2.7.15 interperter while I'm inside a virtual enviroment. I therefore deactivated the virtual env and build the ROS package again. When I then try to launch the ROS package from within in the virtual enviroment I get the following error:

process[gqcnn/grasp_planner-2]: started with pid [1701]
Traceback (most recent call last):
  File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 43, in <module>
    import rospy
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/__init__.py", line 49, in <module>
    from .client import spin, myargv, init_node, \
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/client.py", line 52, in <module>
    import roslib
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslib/__init__.py", line 50, in <module>
    from roslib.launcher import load_manifest
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslib/launcher.py", line 42, in <module>
    import rospkg
ImportError: No module named rospkg

This again is caused by the rospackages being installed in the global python interperter. This behavoir is explained on this topic. I tried to solve this by running the following command:

pip install rospkg

However, I now got a lot of other errors. So I'm unsure if I'm using the package in the right way?

python 3.7.3 conda environment

In conda I got similar errors due to the fact that the ROS packages are installed in the global python 2.7 interpreter. I tried solving some of them using the pip install rospkg command. However, due to the fact that the ROS node is written in python 2.7 (no python3.7 support for ROS kinetic), I get the following error:

Traceback (most recent call last):
  File "/opt/ros/kinetic/bin/roslaunch", line 34, in <module>
    import roslaunch
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/__init__.py", line 51, in <module>
    from . import param_dump as roslaunch_param_dump
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/param_dump.py", line 40, in <module>
    import roslaunch.config
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/config.py", line 45, in <module>
    import rospkg.distro
  File "/opt/conda/envs/tf-gpu/lib/python3.7/site-packages/rospkg/distro.py", line 44, in <module>
    import yaml
  File "/opt/conda/envs/tf-gpu/lib/python3.7/site-packages/yaml/__init__.py", line 374
    class YAMLObject(metaclass=YAMLObjectMetaclass):
                              ^
SyntaxError: invalid syntax

Build all catkin_packages (Autolab_core, perception, visualization, gqcnn & meshrender) in global 2.7 python

I also tried building all berkleyautomation packages using catkin build and installing all the python requirements of each packages using the setup.py files. In this case I also keep running into dependencies and errors. After running the setup.py files the following packages appeared to be missing:

ipython
skimage==0.14

After solving for all the dependencies I get the following error:

Traceback (most recent call last):
  File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 46, in <module>
    from perception import (CameraIntrinsics, ColorImage, DepthImage, BinaryImage,
  File "/catkin_ws/devel/lib/python2.7/dist-packages/perception/__init__.py", line 35, in <module>
    exec(__fh.read())
  File "<string>", line 8, in <module>
  File "/catkin_ws/src/perception/perception/camera_intrinsics.py", line 14, in <module>
    from .image import DepthImage, PointCloudImage
  File "/catkin_ws/src/perception/perception/image.py", line 27, in <module>
    import skimage.morphology as morph
  File "/usr/local/lib/python2.7/dist-packages/skimage/__init__.py", line 167, in <module>
    from .util.dtype import (img_as_float32,
  File "/usr/local/lib/python2.7/dist-packages/skimage/util/__init__.py", line 8, in <module>
    from .arraycrop import crop
  File "/usr/local/lib/python2.7/dist-packages/skimage/util/arraycrop.py", line 8, in <module>
    from numpy.lib.arraypad import _validate_lengths
ImportError: cannot import name _validate_lengths

I have yet to solve this issue but I first wanted to check if I'm not installing the GQCNN package the wrong way.

Question

Can you maybe elaborate on how the ROS package should be used? Do I need to install the python packages in order to use the ROS package? And if so in which environment do I need to install the python packages? And in that case which python version do I need to use? Can I use it with the python 3.7 packages out of the box or do I need to use the multiprocessing library and the socket library in order to achieve this?

Thanks a lot in advance,

visatish · Answer 1 · Mon Jun 24 2019 06:19:09 GMT+0800 (China Standard Time)

Hi @rickstaa,

Sorry to hear that you're having problems with the ROS setup. As a disclaimer, we do not use ROS internally as there is significant communications overhead, and ROS has turned out to be unnecessarily involved in general haha. Thus, the ROS package was really just created in case others wanted to use it, and thus it is a bit buggy due to a lack of rigorous testing. I can share with you what I normally do to get it running:

ROS Kinetic has been installed system-wide on the development machine.

Create a new virtual env (I would stick to Python2.7 here because I actually never got around to testing the ROS capabilities on Python3. I will do so when I get a chance.). This virtual environment should have access to system-wide packages. (I believe it's the default behaviour, but I also think that it can be disabled.)
Install all packages through the appropriate requirements.txt. (You are correct that this needs to be done beforehand since ROS normally uses rosdep for dependencies but we didn't want to go with that since some libraries aren't supported in their registry. I will update the installation instructions.)
Run catkin make and source devel/setup.bash.
You might also have to pip install some libraries here and there like rospkg and rospy. This is just due to a lack of thoroughness on my part. I will update the installation instructions for this too.

Once you try this, let me know if there are still any issues and we can go from there!

Thanks,
Vishal

Rick Staa · Answer 2 · Mon Jun 24 2019 19:44:40 GMT+0800 (China Standard Time)

@visatish Thanks a lot for your answer. I will do some testing this afternoon and report the result. What do you use internally to compute the grasps and control the robot? My plan was now to build a C++ ROS node that spawns your python 3 detection algorithm as a new process and communicate through a shared memory address or UTP sockets.

visatish · Answer 3 · Tue Jun 25 2019 00:39:41 GMT+0800 (China Standard Time)

@rickstaa Internally we simply query the Python policy. The returned actions have a property called grasp which has a function pose() that returns the grasp pose in 3D camera space. You will need to transform to 3D robot space using a known transformation that can be calibrated with a chessboard. So you would execute something like this in code:

T_grasp_camera = action.grasp.pose()
T_grasp_robot = T_camera_robot * T_grasp_camera

For controlling the robot, we use an ABB YuMi and have an open-source Python library here. You will probably have to find something similar for the Panda. Unfortunately, we don't have the bandwidth to help with motion planning itself.

Rick Staa · Answer 4 · Wed Jun 26 2019 22:54:35 GMT+0800 (China Standard Time)

@visatish I did some testing within a python2.7 virtual environment. Ilke you stated your ROS package seems to work inside a python2.7 virtual environment. While following your steps I, however, encountered some errors. For documentation purposes I reported the errors and the solutions to these errors below:

Problems while building the catkin package

No module named em

Errors     << gqcnn:make /catkin_ws/logs/gqcnn/build.make.000.log                                                                                                                                                 
Traceback (most recent call last):
  File "/opt/ros/kinetic/share/gencpp/cmake/../../../lib/gencpp/gen_cpp.py", line 41, in <module>
    import genmsg.template_tools
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/template_tools.py", line 39, in <module>
    import em
ImportError: No module named 'em'
make[2]: *** [/catkin_ws/devel/.private/gqcnn/include/gqcnn/BoundingBox.h] Error 1
make[2]: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "/opt/ros/kinetic/share/gencpp/cmake/../../../lib/gencpp/gen_cpp.py", line 41, in <module>
    import genmsg.template_tools
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/template_tools.py", line 39, in <module>
    import em
ImportError: No module named 'em'
make[2]: *** [/catkin_ws/devel/.private/gqcnn/include/gqcnn/Action.h] Error 1
Traceback (most recent call last):
  File "/opt/ros/kinetic/share/gencpp/cmake/../../../lib/gencpp/gen_cpp.py", line 41, in <module>
    import genmsg.template_tools
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/template_tools.py", line 39, in <module>
    import em
ImportError: No module named 'em'
make[2]: *** [/catkin_ws/devel/.private/gqcnn/include/gqcnn/Observation.h] Error 1
make[1]: *** [CMakeFiles/gqcnn_generate_messages_cpp.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [all] Error 2

Solution

Solved by running pip install empy

Problems while running the ros node

Wrong PYTHONPATH error

After sourcing the ROS setup.bash file and launching the ROS package using the roslaunch gqcnn grasp_planning_service.launch command I get the following errors:

Failed to load Python extension for LZ4 support. LZ4 compression will not be available.
Traceback (most recent call last):
  File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 46, in <module>
    from perception import (CameraIntrinsics, ColorImage, DepthImage, BinaryImage,
  File "/catkin_ws/devel/lib/python3/dist-packages/perception/__init__.py", line 35, in <module>
    exec(__fh.read())
  File "<string>", line 8, in <module>
  File "/catkin_ws/src/perception/perception/camera_intrinsics.py", line 14, in <module>
    from .image import DepthImage, PointCloudImage
  File "/catkin_ws/src/perception/perception/image.py", line 11, in <module>
    import cv2
ImportError: /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
[gqcnn/grasp_planner-2] process has died [pid 11905, exit code 1, cmd /catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py __name:=grasp_planner __log:=/root/.ros/log/b6dee622-9815-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2.log].
log file: /root/.ros/log/b6dee622-9815-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2*.log

WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
Traceback (most recent call last):
  File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 46, in <module>
    from perception import (CameraIntrinsics, ColorImage, DepthImage, BinaryImage,
  File "build/bdist.linux-x86_64/egg/perception/__init__.py", line 8, in <module>
    
  File "build/bdist.linux-x86_64/egg/perception/camera_intrinsics.py", line 14, in <module>
  File "build/bdist.linux-x86_64/egg/perception/image.py", line 27, in <module>
  File "/usr/local/lib/python2.7/dist-packages/skimage/__init__.py", line 167, in <module>
    from .util.dtype import (img_as_float32,
  File "/usr/local/lib/python2.7/dist-packages/skimage/util/__init__.py", line 6, in <module>
    from .apply_parallel import apply_parallel
  File "/usr/local/lib/python2.7/dist-packages/skimage/util/apply_parallel.py", line 8, in <module>
    import dask.array as da
  File "/usr/local/lib/python2.7/dist-packages/dask/array/__init__.py", line 5, in <module>
    from .core import (Array, block, concatenate, stack, from_array, store,
  File "/usr/local/lib/python2.7/dist-packages/dask/array/core.py", line 29, in <module>
    from . import chunk
  File "/usr/local/lib/python2.7/dist-packages/dask/array/chunk.py", line 73, in <module>
    nancumprod = npcompat.nancumprod
AttributeError: 'module' object has no attribute 'nancumprod'
[gqcnn/grasp_planner-2] process has died [pid 27812, exit code 1, cmd /catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py __name:=grasp_planner __log:=/root/.ros/log/278e867a-9820-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2.log].
log file: /root/.ros/log/278e867a-9820-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2*.log

Solution

This problem is explained in this topic. The solution is to add the virtual environment site-packages folder before the ROS dist-packages folder. This can be done running the following code:

_venv_site_packages=$(find "$VIRTUAL_ENV/$2" -iname site-packages)
export PYTHONPATH="$_venv_site_packages:$PYTHONPATH"

To later undo this the following command can be used:

export PYTHONPATH=$(echo $PYTHONPATH | sed -e "s:$_venv_site_packages\:::")

NameError: global name 'FileNotFoundError' is not defined

Traceback (most recent call last):
  File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 410, in <module>
    grasping_policy = CrossEntropyRobustGraspingPolicy(policy_cfg)
  File "/catkin_ws/src/gqcnn/gqcnn/grasping/policy/policy.py", line 726, in __init__
    GraspingPolicy.__init__(self, config)
  File "/catkin_ws/src/gqcnn/gqcnn/grasping/policy/policy.py", line 310, in __init__
    metric_type, self._metric_config)
  File "/catkin_ws/src/gqcnn/gqcnn/grasping/grasp_quality_function.py", line 1303, in quality_function
    return GQCnnQualityFunction(config)
  File "/catkin_ws/src/gqcnn/gqcnn/grasping/grasp_quality_function.py", line 931, in __init__
    self._gqcnn = get_gqcnn_model().load(self._gqcnn_model_dir)
  File "/catkin_ws/src/gqcnn/gqcnn/model/tf/network_tf.py", line 218, in load
    gqcnn.init_mean_and_std(model_dir)
  File "/catkin_ws/src/gqcnn/gqcnn/model/tf/network_tf.py", line 245, in init_mean_and_std
    except FileNotFoundError:
NameError: global name 'FileNotFoundError' is not defined

Solution

This error is not present when I supply the model_name:=GQCNN-4.0-PJ argument with my roslaunch command.