Real-Time High-Resolution Background Matting
Official repository for the paper Real-Time High-Resolution Background Matting. Our model requires capturing an additional background image and produces state-of-the-art matting results at 4K 30fps and HD 60fps on an Nvidia RTX 2080 TI GPU.
- Usage / Documentation
- Project members
- Community Projects
- [Mar 06 2021] Training script is published.
- [Feb 28 2021] Paper is accepted to CVPR 2021.
- [Jan 09 2021] PhotoMatte85 dataset is now published.
- [Dec 21 2020] We updated our project to MIT License, which permits commercial use.
Model / Weights
Video / Image Examples
- VideoMatte240K (We are still dealing with licensing. In the meantime, you can visit storyblocks.com to download raw green screen videos and recreate the dataset yourself.)
We provide several scripts in this repo for you to experiment with our model. More detailed instructions are included in the files.
inference_images.py: Perform matting on a directory of images.
inference_video.py: Perform matting on a video.
inference_webcam.py: An interactive matting demo using your webcam.
Additionally, you can try our notebooks in Google Colab for performing matting on images and videos.
We provide a demo application that pipes webcam video through our model and outputs to a virtual camera. The script only works on Linux system and can be used in Zoom meetings. For more information, checkout:
Developers in the community has helped us build a web demo. See Community Projects section below.
Usage / Documentation
You can run our model using PyTorch, TorchScript, TensorFlow, and ONNX. For detail about using our model, please check out the Usage / Documentation page.
data_path.pth to point to your dataset. The original paper uses
train_base.pth to train only the base model till convergence then use
train_refine.pth to train the entire network end-to-end. More details are specified in the paper.
- Shanchuan Lin*, University of Washington
- Andrey Ryabtsev*, University of Washington
- Soumyadip Sengupta, University of Washington
- Brian Curless, University of Washington
- Steve Seitz, University of Washington
- Ira Kemelmacher-Shlizerman, University of Washington
* Equal contribution.
A list of projects built by third-party developers in the community. If you have a project to share, fill out this survey.
- Web Demo by Gradio: Matting your own images from your browser.