This repository contains the source code for an end-user robot programming system that can be used for programming a robot manipulator by kinesthetic demonstration; the system includes a customizable help menu and chat assistant to support end-users during programming. The system consists of a front-end web interface and back-end software that runs on Ubuntu 20.04 using ROS Noetic.
The programming system is designed to be customized for use with different robots or different support content. We include instructions on how to customize this system in this README.
- frontend: This directory contains the code for the front-end web interface.
- index.html is the entry point for the front-end system.
- backend: This directory contains the back-end code of the robot programming system.
- The subdirectory prog_support_backend is the ROS package folder containing code that handles user program management and robot motion.
- The subdirectory prog_support_marker is the ROS package folder containing code that handles user program visualization.
The front-end code has been deployed on a system running Ubuntu 20.04 (64-bit). The software was hosted using an Apache server (version 2.4.41) and was run on Mozilla Firefox for Ubuntu Version 117.0 (64-bit).
The back-end code has been deployed on a system running Ubuntu 20.04 (64-bit). The software uses ROS Noetic, which can be installed by following the instructions here.
This programming system requires the use of a robot with kinesthetic teaching capabilities and a gripper; you must have access to code that can activate/deactivate the robot's kinesthetic teaching mode and move the robot and its gripper. Instructions for how to set up the system for different robots that meet the requirements above are available below.
To use the system, you will need to create a catkin workspace that contains the two back-end subdirectories (prog_support_backend and prog_support_marker), as well as the required ROS packages specified above and any packages required for the robot you are using (i.e., robot driver). You will first need to modify the back-end code to work with the robot you will be using and prepare JSON files specifying the content you would like the help menu and chat assistant to display.
The file template.cpp includes starter code for the back-end that handles program management and communication with the front-end interface. You will need to add code to the template for interfacing with the robot you are using (see the comments in the file for instructions). For an example of an instantiation of the template for the Kinova Gen3 7-DOF arm, please see kinova_example.cpp.
Once you are finished modifying the template, add the following lines to CMakeLists.txt inside the prog_support_backend package (remember to replace the placeholder below with your file name):
add_executable(backend src/*replace with name of your file*.cpp)
target_link_libraries(backend ${catkin_LIBRARIES})
add_dependencies( backend ${${PROJECT_NAME}_EXPORTED_TARGETS} )
You will also need to modify the file marker_publisher.launch to specify the tf frames for your robot's base and end effector in the following lines:
<!-- Change the base_frame name based on your robot type-->
<arg name="base_frame" default="base_link"/>
<!-- Change the tool_frame name based on your robot type-->
<arg name="tool_frame" default="tool_frame"/>
Then, recompile your catkin workspace.
To run the system, open a terminal on Ubuntu and run the following scripts:
- Run driver and/or any other necessary code required for your robot.
- On a new tab or terminal, run the following script:
roslaunch prog_support_backend web_communication.launch - On a new tab or terminal, run the following script:
rosrun prog_support_backend backend - On a new tab or terminal, run the following script:
roslaunch marker_package marker_publisher.launch
Then, open up a web browser and pull up the index.html file (or appropriate domain if you have hosted the frontend interface on the web).
To populate the support content available on the help menu and chat assistant, you must create JSON files specifying the content you would like to display. A tutorial on how to create the JSON files is available here. After creating the files, you may upload them on the web interface by clicking the "Upload customization file" buttons. Example JSON files are available for the the help menu here and for the chat assistant here.
Note that if you would like to the programming system with the Kinova Gen3 7-DOF arm, you must compile the following package in your catkin workspace: icl_kortex.
Then, in the steps listed above for running the system, run rosrun prog_support_backend kinova_example instead of rosrun prog_support_backend backend
For additional development questions that are not covered in this README, we welcome developers to open an issue or contact the author at gopika@cs.jhu.edu.