At the workshop, we will provide pre-provisioned SSH-able machines for convenience. However, to get the most out of the tutorial, we recommend to install with your local computer. Please refer to Preparation Doc for more information. All of our work is done in the docker, so it will not leave a trace to your computer.
If you successfully followed the Preparation Doc or get a SSH machine from us, you may run the following command to verify your running environment:
MacOS / Linux
cd ~/CloudRobotics_tutorial
./docker-run.sh
Windows
docker-run.cmd
You should get the following output similar to the following:
$ ./docker-run.sh
Starting with USER: jeffi, UID: 501, GID: 20
jeffi@docker-desktop:~/CloudRobotics_tutorial/fog_ws$
If you got that, congrats! Everything is working. At this point, type CTRL-D to exit.
The talker.py and listener.py files are provided in the tutorial_workspace/fogros2_tutorial folder in the repository. If we wanted, we could run both the talker and listener nodes individually and see the nodes communicating. Instead, we are going to run these nodes using a launch file.
The talker.local.launch.py file is provided in the tutorial_workspace/launch in the repository.
- Start the container again
cd ~/CloudRobotics_tutorial
./docker-run.sh
- Run local launch file
ros2 launch fogros2_tutorial talker.local.launch.py
Here you can see both the talker node publishing and the listener node subscribing.
- Run cloud launch file (using FogROS2 and AWS)
Now, we take the same local launch file and run the talker node on a provisioned AWS cloud instance. FogROS2 handles the provisioning and setup of the cloud instance for us.
The talker.aws.launch.py file is provided in the tutorial_workspace/launch folder in the repository.
ros2 launch fogros2_tutorial talker.aws.launch.py
This process will take a few minutes, and you'll see a lot of information scroll by as FogROS2 provisions the cloud instance and installs all required software and dependencies. Finally, you would see both the cloud node and the local node communicating.
CTRL-C kills the local instance (e.g., listener) the first time and then the cloud instance the second time.
This is a bonus part to run a cloud instance with Segment Anything Model (SAM). Do it only if you are assigned with a physical cloudgripper.
Like in Part 1, we have created sam_server.py and sam_client.py which you can look at in the tutorial_workspace/fogros2_tutorial folder. We will be running these nodes using two launch files: sam.aws.launch.py, which are provided in the tutorial_workspace/launch folder in the repository.
run:
ros2 launch fogros2_tutorial sam.aws.launch.py
In this terminal, we launch a server node on the cloud which subscribes to images of the physical robot, loads a SAM model (in this case, we are using the smallest one: "vit_b"), generates masks from the image and then publishes the generated masks. While waiting for the machine to launch, please open up a new terminal, and proceed with CloudGripper.
We integrate FogROS2 with CloudGripper. To interact with the robot, this part assumes that you have already signed up for the robot. You need the assigned robot ID and a robot access API token.
- Start launch files
In the new terminal, run
cd ~/CloudRobotics_tutorial
./docker-run.sh
9.1 Run only if you ran SAM
export CYCLONEDDS_URI=file:///fog_ws/src/FogROS2/fogros2/configs/cyclonedds.ubuntu.2204.xml
This propagate all the messages from CloudGripper to FogROS2-enabled Cloud.
9.2 In the container, run
cd /fog_ws/src/tutorial_workspace/launch
ros2 launch cloudgripper.launch.py -- robot_name:=robotX
where robotX is the robot ID that you are assigned. In this terminal, we launch a client node which receives images of the robot workspace from CloudGripper, publishes the image, and subscribes to generated masks from the cloud. We both the original image from cloudgripper and then generated masks.
In the new terminal, run
cd ~/CloudRobotics_tutorial
./docker-run.sh
10.1 Run only if you ran SAM
export CYCLONEDDS_URI=file:///fog_ws/src/FogROS2/fogros2/configs/cyclonedds.ubuntu.2204.xml
This propagate all the messages to CloudGripper with the same DDS setup.
and in the container run
ros2 run teleop_twist_keyboard teleop_twist_keyboard
This will enable real-time keyboard control for the Cloudgripper.
- Key Bindings:
- Movement:
i: Move forwardk: Move backwardj: Move leftl: Move right
- Z-Axis Control:
t: Move upb: Move down
- Rotation Control:
u: Rotate lefti: Rotate right
- Gripper Control:
m: Close gripper,: Open gripper
- Movement:
If you have ran SAM and CloudGrippper and both are up and running, you can look in ~/CloudRobotics_tutorial/tutorial_workspace/launch/saved_images for both the original image and the saved mask image.
All the data is automatically collected through fog-rt-x, a cloud based data collection and management. fog_rtx_recorder.py shows an example of collecting data from various topics and store them to the cloud. The website will be statically generated and posted at the end of the workshop at link: https://berkeleyautomation.github.io/CloudRobotics_tutorial/