Copyright (c) 2014 Idiap Research Institute, http://www.idiap.ch Written by Kenneth Funes <kenneth.funes@idiap.ch> 1) OVERVIEW: The rgbd module is intended for streaming, visualizing and 3D processing RGB-D data. It allows to process recorded data (mainly videos) as long as calibration parameters are provided. Optionally, the module also builds on top of the PCL library, such that it allows for connected devices (e.g. a Microsoft Kinect, Primesense Carmine 1.08 and 1.09, etc) through the OpenNI (1 or 2) framework. When using recorded data, we unfortunately can't anticipate all possible variants of file formats, etc. We provide a data sample and direct compatibility with the EYEDIAP database to provide usage examples. It should be possible to modify the module for your specific needs. For this, we refer you to the source code. 2) DEPENDENCIES. This module has the following dependencies: - Python 2.7 - Numpy - Scipy - SWIG - Boost - OpenCV (with python bindings) - Qt 4.8 - PySide - OpenGL - PyOpenGL - [Optional] Point Cloud Library >= 1.7. Either: ** Compiled against PCL's patched version of OpenNI and respective drivers (see PCL documentation); or ** Compiled against OpenNI2 (http://structure.io/openni) 3) COMPILATION 3.1) Default Compilation: The default compilation assumes that PCL/OpenNI is not available. Thus it is mainly intended for recorded data. To compile, go to the folder where you have saved the rgbd module and execute the standard cmake compilation procedure: S1. > mkdir build S2. > cd build S3. > cmake .. S4. > make 3.2) Compilation using PCL/OpenNI: Assuming PCL/OpenNI were installed using the standard procedure from the PCL website, you can then compile the rgbd module by setting the variable INCLUDE_PCL_DEVICE_SUPPORT. To do this, just modify the step S3 above as: S3. > cmake -DINCLUDE_PCL_DEVICE_SUPPORT=ON .. 3.3) Compilation using PCL/OpenNI2: Assuming PCL was compiled against a local installation of OpenNI2 (http://structure.io/openni), then modify step S3 as follows: S3. > cmake -DINCLUDE_PCL_DEVICE_SUPPORT=ON -DUSE_OPENNI2=ON .. We added support to OpenNI2 as this project is being actively maintained by Occipital, whereas the OpenNI future is uncertain (to our knowledge). Notice that the rgbd module supports only one of the two versions (OpenNI or OpenNI2), not both at once. 4) INSTALLATION: Once compiled, add the module directory to your python path, for example, by adding the rgbd module path to the environment variable "PYTHONPATH". 5) EXAMPLES: Two main usage examples are given under the "Test" directory. 5.1) recordedExample.py. For this example, you could download either the video sample provided at http://www.idiap.ch/software/rgbd or the EYEDIAP dataset from http://www.idiap.ch/dataset/eyediap. Once you have either, please modify the Test/data.py file such that it properly selects and refers to the data you want to process. Then, you could simply run and modify it to refer to the specific recording session (in case of using EYEDIAP): > python recordedExample.py We refer you to the source code to notice the format of the input data: normally separated video files for the rgb and depth video, together with appropriate calibration parameter files. For the EYEDIAP dataset, it might be useful to have a look at this document: http://publications.idiap.ch/index.php/publications/show/2805 5.2) deviceExample.py This example makes use of the OpenNI(2)/PCL framework. You should have a compliant device, such as a Microsoft Xbox 360 Kinect, a Primesense Carmine 1.08, 1.09, etc. Please make sure the "pcl_openni_viewer" (or "pcl_openni2_viewer") sample program (provided within PCL) is working and, if that is the case, normally this example should work for your connected RGB-D camera. To execute it, please run: > python deviceExample.py Notice in this case the "source" data is an integer, denoting a device (although at this point it supports a single device), and the calibration information is "None" meaning it is obtained from OpenNI.