RHCR - Remote Haptic Control Robot

RHCR is a scientific initiation project carried out by NTA's Laboratory at UFES.

Our goal is to promote a user-friendly robot controller using virtual reality elements, creating an environment where the user can see, feel and control a robot even if they are not in the same place.

1. Components

1.1. Hardware Components:

Wheeltec Ackermann Robot
- using specific ROS 2 packages provided by Wheeltec.
Meta Quest 2
- using Oculus software provided by Meta.
Novint Falcon
- using libnifalcon library.

1.2. Software Components:

Unity
- using XR and ROS-TCP-Connector packages.
ROS 2
- using ros2-falcon, navigation2, slam_toolbox and ros-tcp-endpoint packages.

2. Installing RHCR:

This project needs two operating systems, a Linux machine to run ROS2 and connect to Novint Falcon, and a Windows machine to run Unity and connect to Quest2.

Below is a step-by-step tutorial on how to prepare each environment:

2.1. Preparing the Linux environment:

Recommended version: Ubuntu 22.04

2.1.1. Configuring network adapters (virtual machines only):

Set network adapter to Bridged mode and ONLY turn on the adapter that the robot is connected to:

2.1.2. Installing ROS and its dependencies:

Install ROS 2 Humble using the official documentation or the script below:

sudo apt update && sudo apt install locales -y
sudo locale-gen en_US en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LANG=en_US.UTF-8
sudo apt install software-properties-common -y
sudo add-apt-repository universe
sudo apt update && sudo apt install curl -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null
sudo apt update && sudo apt upgrade -y
sudo apt install ros-humble-desktop -y

Install Colcon and Rosdep:

sudo apt install python3-colcon-common-extensions -y
sudo apt install python3-rosdep -y
sudo rosdep init
rosdep update

2.1.3 Creating and populating the rhcr workspace:

Create a workspace, install git and clone the ros-packages there:

mkdir -p ~/rhcr_ws/src
cd ~/rhcr_ws/src
sudo apt install git -y
git clone -b ros-packages --recurse-submodules https://github.com/jvmoraiscb/rhcr.git .

Install ros2-falcon drivers:

cd ~/rhcr_ws/src/ros2-falcon/libnifalcon
./install.sh

Install packages depencies:

sudo apt-get install -y \
python3-colcon-common-extensions \
ros-humble-joint-state-publisher \
ros-humble-xacro \
ros-humble-slam-toolbox \
ros-humble-navigation2 \
ros-humble-nav2-bringup \

Build the colcon workspace:

source /opt/ros/humble/setup.bash
cd ~/rhcr_ws
rosdep install -i --from-path src --rosdistro humble -y
colcon build

2.2. Preparing the Windows environment:

Recommended version: Windows 11

2.2.1. Install Git:

Visit git-scm.com/downloads, download 64-bit Git for Windows Setup and install it.
Restart the computer.

2.2.2. Install ROS 2 Humble:

Visit docs.ros.org/en/humble and follow the instructions (if the link is no longer available, this repository has a copy of the installation guide here).
Open a PowerShell terminal with administrator privileges and unblock the startup script:

Set-ExecutionPolicy Unrestricted ; Unblock-File C:\ros2-windows\local_setup.ps1

2.2.3. Install Oculus Software:

Visit meta.com/quest/setup, scroll down to the Quest 2 section, click on Download Software and install it.
Sign in or create a Meta Account and set up your Quest 2.
Go to Settings -> General and enable Unknown Sources and OpenXR Runtime.

2.2.4. Install Unity:

Visit unity.com/download, download and install Unity Hub.
Visit unity.com/releases/editor/archive and find Unity 2020.3.29 version, then click the Unity Hub button and proceed to install the editor.
Choose a folder and clone this branch there.

git clone -b unity-project https://github.com/jvmoraiscb/rhcr.git

2.2.4 Set the ROS domain ID that you will use across all packages:

Open a PowerShell terminal with administrator privileges and set as a global environment variable:

setx /m ROS_DOMAIN_ID 42

3. Running RHCR:

Before running the project, it's a good idea to check that both computers are "seeing" each other (just ping them).

3.1. Running in the Linux environment:

Disconnect Novint Falcon from host and connect to virtual machine (virtual machines only):
Open a terminal, source rhcr workspace, set the domain id to the same one used previously and run ros2-falcon (follow the terminal instructions to calibrate the controller):

source ~/rhcr_ws/install/setup.bash
ROS_DOMAIN_ID=42 ros2 run ros2-falcon main

Open another terminal, source rhcr workspace, set the domain id to a DIFFERENT one from the one used previously (only at this step) and launch rhcr:

source ~/rhcr_ws/install/setup.bash
ROS_DOMAIN_ID=24 ros2 launch rhcr start.launch.py

3.2. Running in the Windows environment:

3.2.1. Configuring the network:

Disable Windows Firewall:
Go to Windows Advanced network settings and disable all adapters that are not connected to the same network as the robot (same way as in virtual machine):

3.2.2. Connecting to the robot and launching its default package:

Connect to the same network as the robot:
Open a PowerShell terminal and connect to robot through ssh protocol:
Set the domain id to the same one used previously and launch the robot's default package:

ROS_DOMAIN_ID=42 ros2 launch robot_package robot_package.launch.py

Replace robot_package for the robot ros2 package name, and robot_package.launch.py for the launch file name.

3.2.3. Running the unity project:

Open a PowerShell terminal, run ros2 startup script and open the project with all variables loaded.

C:\ros2-windows\local_setup.ps1 ; Start-Process -FilePath '<path\to\Unity.exe>' -ArgumentList '-projectPath "<path\to\unity-project\>"'

Replace <path\to\Unity.exe> for the path to the Unity executable, and <path\to\unity-project> for the path to the project folder.

Go to RemoteHapticControlRobot -> RosTcpConnector -> VirtualSensors and enter the IP address of the Linux environment in the Ros IP Address field.

jvmoraiscb / rhcr