This is a ROS package for Intel realsense D435i with 3-DOF Manipulator robot that can be used for Indoor Mapping and localization of objects in the world frame with an added advantage of the robot's dexterity. The 3-DOF Manipulator is a self-built custom robot where the URDF with the depth sensor is included. The package covers the Rosserial communication with Arduino nodes or I2C with the Jetson Nano to control the robot's Joint States and PCL pipelines required for autonomous mapping/Localization/Tracking of the objects in real-time.
The Robot's URDF model represents the exact simplified version of the actual model with measured offsets from joint to joint.
The RRR Robot is built using standard servo 3 x Clamps,1 x Long U Mount and 45deg U Mount.
cd catkin_ws/src
git clone https://github.com/fazildgr8/realsense_bot -b jetson
cd catkin_ws/src
git clone https://github.com/fazildgr8/realsense_bot -b master
Launch the Robot in RVIZ with Manual Joint Control.
This Launch file starts the Robot state publisher which updates the Transformation tree using the Joint states from Joint state publisher. The Robot can be Controlled manually by publishing the Joint states message to the Joint State Controller.
For Jetson Nano Implementation execute the below mentioned
roslaunch realsense_bot robot_visualize_control_jetson.launch
For PC+Arduino Implementation execute the below mentioned
roslaunch realsense_bot robot_visualize_control.launch
Launch the IntelRealsense Camera node to start the PCL Perception
This Launch file starts the IntelRealsense camera node which publishes all the messages w.r.t to the Realsense Camera.
roslaunch realsense2_camera rs_camera.launch filters:=pointcloud align_depth:=true depth_width:=640 depth_height:=480 depth_fps:=30 color_width:=640 color_height:=480 color_fps:=30
Launch the Yolo detector node to produce Object Detection and Bounding Boxes Messages
roslaunch yolo_detect.launch
Launch the node which deprojects the 2D detected object in 3D using rs2_pixel_to_point_deprojection(...) with the Node /yolo_pixel_to_3Dpoint
rosrun realsense_bot yolo_pixel_to_3Dpoint.py
Launch the node which follows and tracks the Object using the 3-DOF Manipulator
rosrun realsense_bot object_follow.py
Launch the below mentioned seperately to begin RTAB-Mapping with Madwick IMU Filter and RVIZ representation of the Cloud Map.
roslaunch realsense_bot robot_visualize_control.launch
roslaunch realsense_bot realsense_mapping.launch
- Direct Implementation - The setup for PC with Arduino control uses serial communication to control the servos through the Arduino which is capable producing PWM signals.
- Jetson Nano Standalone Implementation - The setup and installation differs for the Jeston boards for using it standalone as GPIO pins present would not be able to produce PWM signals for the servo motors instead we will be using a Servo Controller board which uses I2C protocol to control the Servos. Added to it RTAB Mapping is currently unstable with Realsense D435i in Jetson Nano due to its instability in publishing PCL data.
sudo apt-get install ros-$ROS_DISTRO-realsense2-camera
sudo apt-get install ros-$ROS_DISTRO-realsense2-description
sudo apt install ros-$ROS_DISTRO-image-transport-plugins
sudo apt install ros-$ROS_DISTRO-rtabmap-ros
sudo apt install ros-$ROS_DISTRO-rosserial
sudo apt install ros-$ROS_DISTRO-joint-state-publisher-gui
pip install pyrealsense2
pip install opencv-python
cd catkin_ws/src
git clone https://github.com/ccny-ros-pkg/imu_tools -b $ROS_DISTRO
cd ..
catkin_make
Installing darknet_ros package which is the YoloV3 implementation as a ROS Node.
On Cuda set up systems make sure to modify the CMakeLists.txt within the darknet_ros package to support compute capability of your systems. For details please go through
darknet_ros Official Package
cd catkin_ws/src
git clone https://github.com/ccny-ros-pkg/imu_tools -b $ROS_DISTRO
git clone --recursive git@github.com:leggedrobotics/darknet_ros.git
cd ..
catkin_make
The Arduino code file required for flashing can be found in arduino/ros_servo_3dof.ino
Connect the three Servo's signal pin the to 9 10 11 pins of the Arduino. The pin configuration can be modified within ros_servo_3dof.ino.
[To Be Updated]
