PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

Created by Yu Xiang at RSE-Lab at University of Washington and NVIDIA Research.

Introduction

We introduce PoseCNN, a new Convolutional Neural Network for 6D object pose estimation. PoseCNN estimates the 3D translation of an object by localizing its center in the image and predicting its distance from the camera. The 3D rotation of the object is estimated by regressing to a quaternion representation. arXiv, Project

License

PoseCNN is released under the MIT License (refer to the LICENSE file for details).

Citation

If you find PoseCNN useful in your research, please consider citing:

@inproceedings{xiang2018posecnn,
    Author = {Xiang, Yu and Schmidt, Tanner and Narayanan, Venkatraman and Fox, Dieter},
    Title = {PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes},
    Journal   = {Robotics: Science and Systems (RSS)},
    Year = {2018}
}

Environment

本人系统环境：

Ubuntu 16.04
Tensorflow 1.8（from source）
Python 2.7
Cuda 10.0 & cuddn 7.3.1

Setting

1.搭建虚拟环境

第一步，创建专属于PoseCNN的虚拟环境，之后install的包都在此虚拟环境中。虚拟环境的好处不用多说了吧，反正对Ubuntu系统的折腾越少越好！！！我用 conda 创建的环境：

conda create -n posecnn python=2.7 激活环境：
conda activate posecnn 如果不用这个环境，记得deactivate：
conda deactivate posecnn

2.pip install

pip install opencv-python

如果不行试一下： sudo apt-get install libopencv-dev

pip install mock enum34
pip install matplotlib numpy keras Cython Pillow easydict transforms3d
pip install OpenEXR
sudo apt-get install libsuitesparse-dev libopenexr-dev metis libmetis-dev

3.TensorFlow

注意一定要从源码安装，虽然很繁琐，但是经过实践证明，pip install安装出来的TensorFlow不好用。。此外，使用gcc 4.8和g++ 4.8对后续的依赖包进行编译。

sudo apt-get install gcc-4.8

sudo apt-get install g++-4.8

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 10

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 30

sudo update-alternatives --config gcc 输入选择 1

sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 10

sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5 30

sudo update-alternatives --config g++ 输入选择 1

测试一下gcc和g++的版本，显示4.8就更换完毕了:

gcc --version

g++ --version

接下来安装bazel，并选择0.10.0版本，本文选择下载sh文件进行安装，

下载地址：https://github.com/bazelbuild/bazel/releases/download/0.10.0/bazel-0.10.0-installer-linux-x86_64.sh 下载好之后，安装：

chmod +x bazel-0.10.0-installer-linux-x86_64.sh 修改文件权限

./bazel-0.10.0-installer-linux-x86_64.sh --user 进行安装接着添加环境变量：

gedit ~/.bashrc

export PATH="$PATH:$HOME/bin"

下面下载安装TensorFlow：

git clone https://github.com/tensorflow/tensorflow.git

cd tensorflow

git checkout r1.8

./configure 这一步，配置文件会问很多问题，对应回答y/n即可:

注意 Python 及其sitepackage的路径要与你之后环境路径相对应比如我在posecnn虚拟环境中运行的话，我的python路径就是 .../.conda/env/posecnn/bin/python 大部分都选择n，但是询问cuda时，要根据你的电脑实际选择

然后编译源文件：

bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package 生成安装包：

bazel-bin/tensorflow/tools/pip_package/build_pip_package ~/software/tensorflow 最后安装：

pip install /tmp/tensorflow_pkg/tensorflow-1.8.0-cp27-cp27mu-linux_x86_64.whl 至此，TensorFlow的源码安装大功告成，可以import测试一下。

4.Eigen

wget https://bitbucket.org/eigen/eigen/get/3.3.0.zip
# 提取解压压缩包
# 重命名文件夹为eigen
cd eigen
mkdir build && cd build
cmake ..
make
sudo make install

5.Nanoflann

wget https://github.com/jlblancoc/nanoflann/archive/ad7547f4e6beb1cdb3e360912fd2e352ef959465.zip
# 提取解压压缩包
# 重命名文件夹为nanoflann
sudo apt-get install build-essential cmake libgtest-dev
cd nanoflann
mkdir build && cd build && cmake ..
make && make test
sudo make install

6.Pangolin

wget https://github.com/stevenlovegrove/Pangolin/archive/1ec721d59ff6b799b9c24b8817f3b7ad2c929b83.zip
# 提取解压压缩包
# 重命名文件夹为Pangolin
cd Pangolin
# Add folowing line to the CMakeLists.txt:
# add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)
mkdir build
cd build
cmake ..
cmake --build .

7.Boost

wget https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2
# 提取解压压缩包
# 重命名文件夹为boost
cd boost
./bootstrap.sh
sudo ./b2
sudo ./b2 install

8.Sophus

wget https://github.com/strasdat/Sophus/archive/ceb6380a1584b300e687feeeea8799353d48859f.zip
# 提取解压压缩包
# 重命名文件夹为Sophus
cd Sophus
mkdir build && cd build
cmake ..
make
sudo make install

9.NLOPT

wget https://github.com/stevengj/nlopt/archive/74e647b667f7c4500cdb4f37653e59c29deb9ee2.zip
# 提取解压压缩包
# 重命名文件夹为nlopt
cd nlopt
mkdir build
cd build
cmake ..
make
sudo make install

至此，所有依赖包配置完毕，下面针对源代码进行编译运行。

10.Compile lib/kinect_fusion

先注释掉/usr/local/cuda/include/crt/common_functions.h的第75行 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported. Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead." 因为这个issue 要是只读权限无法修改，就用sudo chmod 777 /usr/local/cuda/include/crt/common_functions.h修改一下权限。

cd kinect_fusion
mkdir build
cd build
cmake ..
make

编译完记得取消注释刚刚的common_functions.h第75行

11.Compile lib/synthesize

cd ..
cd ..
cd synthesize
mkdir build
cd build
cmake ..
make

Compile the new layers under $ROOT/lib we introduce in PoseCNN. （注意下面的$ROOT要换成你实际的PoseCNN代码路径！！！）

cd $ROOT/lib
sh make.sh

run python setup: python setup.py build_ext --inplace
Add pythonpaths
Add the path of the built libary libsynthesizer.so to python path

export PYTHONPATH=$PYTHONPATH:$ROOT/lib:$ROOT/lib/synthesize/build

12.下载数据集

下载YCB-Video数据集，提取码52xx，解压生成：PoseCNN/data/YCB
下载SUN2012数据集,解压生成：PoseCNN/data/SUN2012/data
下载ObjectNet3D数据集,解压生成：PoseCNN/data/ObjectNet3D/data

至此，环境配置完毕。接下来直接贴出原作者步骤：

Running the demo

Download our trained model on the YCB-Video dataset from here, and save it to $ROOT/data/demo_models.

run the following script

./experiments/scripts/demo.sh # 默认用0号GPU运行!
# 或者
./experiments/scripts/demo.sh --gpuid 1 # 指定1号（也可以选择你喜欢的GPU）运行空格很重要!

Running on the YCB-Video dataset

Download the YCB-Video dataset from here.数据集上一步已经下好了，这一步不用管~
Create a symlink for the YCB-Video dataset (the name LOV is due to legacy, Learning Objects from Videos) 建立软连接，让代码知道你数据集放哪了。
```
cd $ROOT/data/LOV
ln -s $ycb_data data
ln -s $ycb_models models
```

Training and testing on the YCB-Video dataset

cd $ROOT

# training
./experiments/scripts/lov_color_2d_train.sh $GPU_ID

# testing
./experiments/scripts/lov_color_2d_test.sh $GPU_ID

更多可以看下面的参考链接，很详细。更多多的希望通读代码！通读代码！通读代码！

参考：

PoseCNN RSE-Lab，RSE-Lab
PoseCNN GitHub代码，yuxng
YCB-Video数据集下载，提取码52xx，wangg12
PoseCNN代码实现大纲，Kaju-Bubanja
PoseCNN代码实现详细，Luedeke
《论文笔记——PoseCNN》，XJTU_Bugdragon

hatimwen / PoseCNN