hsp-iit / roft

Real-time Optical Flow-aided 6D Object Pose and Velocity Tracking

ROFT

ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking

IEEE Robotics and Automation Letters, vol. 7, no. 1, pp. 159-166, Jan. 2022

Paper | arXiv | Video

Reproducing the experiments

We support running the experiments on the Fast-YCB dataset via the provided Docker image.

If you want to install the repository manually, please refer to the recipe contained in the Dockerfile.

1. Pull the docker image:

   docker pull ghcr.io/hsp-iit/roft:latest

2. Launch the container:

   docker run -it --rm --user user --env="DISPLAY" --net=host --device /dev/dri/ ghcr.io/hsp-iit/roft:latest

   If an NVIDIA GPU is adopted, please use instead:

   docker run -it --rm --user user --env="DISPLAY" --net=host -e NVIDIA_DRIVER_CAPABILITIES=all -v /tmp/.X11-unix:/tmp/.X11-unix --gpus all --runtime=nvidia ghcr.io/hsp-iit/roft:latest

3. Update and build the project:

   cd /home/user/roft
   git pull
   cd build
   make install

4. Download and extract the accompanying data (Fast-YCB dataset and pre-evaluated results) and the YCB-Video model set:

   cd /home/user/roft
   bash tools/download/download_results.sh
   bash tools/download/download_fastycb.sh
   bash tools/download/download_ycb_models.sh

5. Initialize the datasets:

   cd /home/user/roft
   bash test/init.sh

6. Run the experiments (optional):

   cd /home/user/roft
   bash test/run_paper_experiments

   The accompanying data contains the pre-evaluated results. If desired, the results can be re-evaluated using the above command.

7. Run the evaluation:

   cd /home/user/roft
   bash evaluation/run_paper_evaluation

8. Visualize the results: The results on the Fast-YCB dataset (Table I, II, IV and Figure 3) can be found in /home/user/roft/evaluation_output:

   • tableI.pdf
   • tableII.pdf
   • tableIV.pdf
   • Fig3_*.png

   The docker image provides evince and eog in order open pdf and png files, respectively.

In order to run part of the provided software it could be required to temporarily execute xhost + in a console outside of Docker in order to allow the container accessing the X server facilities. The command can be run even after the container has been already launched.

Support for reproducing the experiments on the HO-3D dataset will be added in the near future.

Instructions on how to use the ROFT library in external C++ projects and how to execute ROFT on custom datasets will be added in the near future.

Citing ROFT

If you find the ROFT code useful, please consider citing the associated publication:

@ARTICLE{9568706,
author={Piga, Nicola A. and Onyshchuk, Yuriy and Pasquale, Giulia and Pattacini, Ugo and Natale, Lorenzo},
journal={IEEE Robotics and Automation Letters},
title={ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking},
year={2022},
volume={7},
number={1},
pages={159-166},
doi={10.1109/LRA.2021.3119379}
}

and/or the repository itself by pressing on the Cite this respository button in the About section.

The pre-evaluated results on the Fast-YCB dataset are stored within the IIT Dataverse and identified by the following .

Generate optical flow frames for a custom dataset

If a custom dataset, using the same format as Fast-YCB, is available, the optical flow frames can be generated by:

1. Enabling the option BUILD_NVOF when building using CMake
2. Using the ROFT-of-dumper executable (synopsis available via CLI)

Note: a system with a optical flow-enabled NVIDIA GPU is required. Moreover, OpenCV should be compiled with the cudaoptflow contribution module.

Maintainer

This repository is maintained by:

	@xenvre