Unwrapping OmniDirectional Images

Copyright 2010-2014 METU, Middle East Technical University, Informatics Institute

This file is part of "Unwrapping OmniDirectional Images" Library.

"Unwrapping OmniDirectional Images" is free software: 
you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

"Unwrapping OmniDirectional Images" is distributed in the hope 
that it will be useful, but WITHOUT ANY WARRANTY; without 
even the implied warranty of MERCHANTABILITY or FITNESS FOR A 
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License.
If not, see <http://www.gnu.org/licenses/>. 
Please contact Alptekin TEMIZEL for more info about 
licensing atemizel@metu.edu.tr.

Introduction

The transformation of Omni-directional (OD) images into perspective-view images is an intensive task which takes significant time especially when the image resolution is high. Recently, Graphics Processing Units (GPU) have been started to be used for process intensive tasks, including various image processing algorithms, and significant speed-ups against CPU counterparts have been reported. In this paper, we propose parallel implementation of un-warping OD images on GPU, implemented using Compute Unified Device Architecture (CUDA). The performance of the proposed parallel algorithm has been analysed on two different GPU devices against the performance of its sequential version executed on the CPU. Our test results indicate a speed-up of up to 32 times by the execution of the parallel algorithm on the GPU. The performance analysis also indicates that the memory copy operations take significant amount of time on the GPU implementation. Hence we further analysed these copy operations and suggested optimizations for them.

Even though the uploaded word document had been written in the year 2010; all parameters passed to the new API remained the same. Please check out that file for construction parameters of "OmniOperations" class.

System Requirements

Although your requirements may vary (due to many reasons such as the quality of the image your recorder device outputs), following table illustrates the platform "Unwrapping OmniDirectional Images" had been tested. Running it on a Platform with "System Requirements" equal to the one given in the table below or higher is recommended. Only important thing that should be taken into consideration is that Kepler Specific instructions and keywords are used in "Unwrapping OmniDirectional Images" library; hence using a CUDA-powered GPU with version of 3.0+ is a mandotary.

Id	Requirement	Test Platform
1	OS	Windows 7 x64
2	CPU	Intel i7 4770K 3.5 GHz
3	GPU	NVIDIA Tesla K40c Compute Capability 3.5, 15 SMs 192 Cores per SM, 2 Async Copy Engine, Hyper-Q Enabled
4	RAM	32GB DDR3
5	Serial Computer Expansion Bus	PCIe 2.1
6	CUDA Toolkit	6.0
7	CUDA Driver	6.0
8	CUDA Runtime	6.0
9	OpenCV	2.4.9

Installation Details

There is no specific installer for now. All you require (except the ones defined in "Systems Requirements" section) are Visual Studio, CUDA Toolkit and OpenCV for generating dll and executable files. All project specific files in uploaded solution were created by Visual Studio 2012. Please ensure that you installed NVIDIA Nsight, right after you'd installed Visual Studio IDE; otherwise your projects will not be detected as CUDA projects.
There are two separate projects in the solution :

One for "Unwrapping OmniDirectional Images" Library which produces dll and lib files.
One for Testing the Library which produces an executable file.

Paths to "Unwrapping OmniDirectional Images" Library's header and library files for test project were defined with respect to the solution's root folder. However, for OpenCV repeat the steps given below for both projects :

Select the Project.
Click on Project->Properties->Configuration Properties->C/C++->General->Additional Include Directories.
Specify where the include folder for OpenCV Library is.
Click on Project->Properties->Configuration Properties->Linker->General->Additional Library Directories.
Specify where lib folder for OpenCV Library is. All Required library files are already defined in Input sub-section.

As for CUDA support :

Select the Project for "Unwrapping OmniDirectional Images" Library which is "FlatOmniLib".
Click on Project->Properties->Configuration Properties->CUDA C/C++->Device->Code Generation.
You should specify compute capability and SM architecture versions to optimize your code in accord with the device you have. For us, they were both equal to 3.5

Usage

Four different samples (varying in size) of the same omnidirectional image are uploaded for testing. Files are as follows (the original image was taken by Ladybug Omnidirectional Camera System) :

39,894222,32,815392_Original.jpg : Original Image 5184x3456
39,894222,32,815392_Quartered.jpg : Original Image 2592x1728
39,894222,32,815392_OneOver16.jpg : Original Image 1296x864
39,894222,32,815392_Quadrupled.jpg : Original Image 10368x6912

To run the application :

Open a Command Line.
Change Directory to $(SolutionDir)\x64\Release.
Type : TestOmniApp.exe "39,894222,32,815392_Original.jpg"
Press on Enter.

You should be able to see that "tryMeBoy.jpeg" file has just been generated in exactly the same folder as where the executable file resides in.

In the Main.cpp file, there you should see two parameters were passed to the method "unwrap".

If the first parameter is an empty string; then it is user's responsibility to save the file using the C++ pointer returned by this method.
Second parameter is a boolean value. Since there might be some visual distortions due to the unwrapping; you should tell the API whether it should run a smoothing filter on the resulting image or not (smoothing is run on the GPU as well).

Notes

Keep in mind that a Tesla card was used during the test phase for it eliminates the overhead brought by display drivers like WDDM. This is valid for Windows Platforms only in the case you run "Unwrapping OmniDirectional Images" Library on Windows Vista and beyond. There might be some temporal solutions proposed on NVIDIA's web site to breach WDDM like drivers.
Note that, CUDA Texture Objects made running multiple instances of the new API in separate thread contexts possible. Now users may create several instances of the "OmniOperations" class so that they are able to unwrap different images from different resources(cameras) in separate threads (via OpenMP, for example) concurrently.
As of yet, only CV_8UC1 (gra level), CV_8UC3 (bgr), and CV_8UC4 (bgra) image types are supported by the API.

iliTheFallen / flatOmniLib