xdxuefei / mpegflow

A tool for easy extraction of motion vectors stored in video files

Information & Contact

We release mpegflow for easy extraction of MPEG-flow (motion vectors) from video files along with a visualization tool vis, both under the MIT license. We provide Makefiles for Linux / Windows and distribute binary releases.

Please submit bugs on GitHub directly. For any other question, please contact Vadim Kantorov at [e-mail] or [e-mail].

If you use this code, please cite our work:

@inproceedings{kantorov2014,
      author = {Kantorov, V. and Laptev, I.},
      title = {Efficient feature extraction, encoding and classification for action recognition},
      booktitle = {Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2014},
      year = {2014} }

Below is a visualization of some flow maps of MPI Sintel's alley_1 sequence:

Source	3	15	50
mpegflow
Ground truth

mpegflow

The tool accepts a video file path as command-line argument and writes MPEG-flow (motion vectors) to standard output. By default, the tool outputs the motion vectors arranged in two matrices - dx and dy.

Command-line options:

Option	Description
--help or -h	will output the usage info
--raw	will prevent motion vectors from being arranged in matrices
--grid8x8	will force fine 8x8 grid
--occupancy	will append occupancy matrix after motion vector matrices
--quiet	will suppress debug output

vis

The tool accepts a video file path and a dump directory as command-line arguments, output of mpegflow on standard input and saveson disk the visualization of motion vectors overlayed on video frames.

Command-line options:

Option	Description
--help or -h	will output the usage info
--occupancy	will expect occupancy information from mpegflow and will visualize it as well

Examples

• To extract motion vectors:

  ./mpegflow examples/mpi_sintel_final_alley_1.avi > examples/alley_1.txt

• To visualize motion vectors:

  mkdir -p examples/vis_dump && ./mpegflow examples/mpi_sintel_final_alley_1.avi | ./vis examples/mpi_sintel_final_alley_1.avi examples/vis_dump

Feel free to copy-paste and run the examples above. More runnable examples are in examples/extract_motion_vectors.sh and examples/vis_motion_vectors.sh. Feel free to use vis.cpp and examples/vis_hue.m as examples of parsing mpegflow output. examples/vis_hue can also be used to produce hue flow visualizations like above.

Building from source

mpegflow depends only on a recent FFmpeg, vis depends on FFmpeg, OpenCV and libpng. The tools are known to work with FFmpeg 2.7.2 and OpenCV 2.4.11. We strongly recommend running the following snippet to install the dependencies:

cd dependencies
bash install_ffmpeg_here_linux.sh  # to install ffmpeg
bash install_opencv_here_linux.sh  # to install opencv
cd ..                              # to move back from dependencies directory

Once the dependencies are visible to g++, run:

make     # to build mpegflow
make vis # to build vis

To build the tools on Windows:

1. Follow dependencies\install_dependencies_here_windows.txt

2. Open VS2015 x64 Native Tools Command Prompt (VS2015 Community Edition will work) and run:

nmake mpegflow.exe FFMPEG_DIR=dependencies\ffmpeg-3.0.1-win64-dev\ffmpeg-3.0.1-win64-dev

nmake vis.exe OPENCV_DIR=dependencies\opencv-3.1.0\opencv\build\x64\vc14

3. The Windows build is not fully static. You need to keep `avutil-54.dll`, `avformat-56.dll`, `avcodec-56.dll`, `swresample-2.dll` (for **mpegflow**) and `opencv_world310.dll` (for **vis**) in the same directory as the binary. Note that the instructions and the Makefile assume x64 machine architecture.