andrewkchan / pytorch_mesh_renderer

Some implementations of differentiable 3D mesh renderers using PyTorch with examples

Introduction

This repository contains implementations of two differentiable 3D mesh renderers using PyTorch:

• mesh_renderer: A port of Google's tf_mesh_renderer from Tensorflow to PyTorch. Based on the barycentric formulation from Genova et al. 2018 "Unsupervised training for 3d morphable model regression."
• soft_mesh_renderer: An alternate implementation of SoftRas that I built for my own learning. Based on the probabilistic rasterization formulation by Liu et al. 2019 "Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning".

Setup

1. Create a virtual environment with python3 -m venv env
2. Activate it with source env/bin/activate
3. Install external dependencies with pip install -r requirements.txt

Some additional setup is required to use the optimized kernel for the barycentric renderer. See docs for more.

Testing

Tests are included for both renderers.

• mesh_renderer: See mesh_renderer docs for how to run these tests.
• soft_mesh_renderer: See soft_mesh_renderer docs for how to run these tests.