junyanz / light-field-video

Light field video applications (e.g. video refocusing, focus tracking, changing aperture and view)

Light Field Video Viewer

[Project] [Video] [Paper]

C++ implementation for Light field video applications (including video refocusing, focus tracking, changing aperture and view) as described in the following paper:

Light Field Video Capture Using a Learning-Based Hybrid Imaging System
Ting-Chun Wang, Jun-Yan Zhu, Nima Khademi Kalantari, Alexei A. Efros, and Ravi Ramamoorthi
In ACM Transactions on Graphics (SIGGRAPH 2017)

Prerequisites

• Linux. (The code should work on OSX/Windows with minimal modification)
• OpenMP, OpenCV, QT 5.7, cmake.

Getting Started

• Clone this repo:

git clone https://github.com/junyanz/lfv_release.git lfv
cd lfv

• Please modify line 5 in CMakeLists.txt to include the QT5 installation path.

• Download light field video data (e.g. cats).

bash data/download_lfv.sh cats

• Compile the code:

mkdir cbuild
cd cbuild
cmake -DCMAKE_BUILD_TYPE=Release  ../
make

• Start the viewer:

./lfv --data_dir ../data/cats/ --scale 2.0

Arguments:

• --data_dir: the directory that stores light field video frames, and disparity maps
• --scale: up-sample the image by scale.

User Interface

Mouse Interaction

• Refocusing: press + left click
• View Change: move the mouse + right click

Control Panel

• Focal Plane: change the focal plane
• Aperture: change the camera aperture
• Focal Type: Fixed focal point or automatic tracking
• Restart (r): restart the system
• Play/Stop (p): play or pause the video
• Quit (q): quit the program

Shortcuts:

• Show/hide the focus point: f
• Enable/disable tracking mode: t

Training/test code

Light field videos:

Please use the script bash data/download_lfv.sh video_name to download the light field video from our website.

cats

train1

train2

dancing

Citation

If you use this code for your research, please cite our paper.

@article{wang2017light,
    author  = {Ting-Chun Wang and Jun-Yan Zhu and Nima Khademi Kalantari and
               Alexei A. Efros and Ravi Ramamoorthi},
    title   = {Light Field Video Capture Using a Learning-Based Hybrid Imaging System},
    journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2017)},
    volume  = {36},
    number  = {4},
    year    = {2017},
}

Cat Paper Collection

If you love cats, and love reading cool graphics, vision, and learning papers, please check out the Cat Paper Collection:
[Github] [Webpage]