Giters

colorful-liyu / awesome-3D-editing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

awesome-3D-editing

Open-source Toolboxes and Foundation Models

Methods Task Github
threestudio T23D Generation Star
DreamGaussian T23D Generation Star

Table of Contents

Paper list (Latest Update: 2023/10/26)

Preliminary: milestones for generation and editing

Classical representation

Title arXiv WebSite Github Pub.
3D Gaussian Splatting for Real-Time Radiance Field Rendering arXiv Website Star SIGGRAPH 2023
Instant Neural Graphics Primitives arXiv Website Star SIGGRAPH 2022

Classical text-3d generation

Title arXiv WebSite Github Pub.
ATT3D: Amortized Text-to-3D Object Synthesis arXiv Website - ICCV 2023
FantASIA3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation arXiv Website Star ICCV 2023
Zero-1-to-3: Zero-shot One Image to 3D Object arXiv Website Star ICCV 2023
Magic3D: High-Resolution Text-to-3D Content Creation arXiv Website - CVPR 2023
Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model arXiv - Star ArXiv, 2023, 10
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image arXiv Website Star ArXiv, 2023, 09
MVDream: Multi-view Diffusion for 3D Generation arXiv Website Star ArXiv, 2023, 08
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization arXiv Website Star ArXiv, 2023, 06
Shap·E: Generating Conditional 3D Implicit Functions arXiv - Star ArXiv, 2023, 05
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures arXiv - Star ArXiv, 2022, 11
DreamFusion: Text-to-3D using 2D Diffusion arXiv Website Star ArXiv, 2022, 09
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation arXiv Website Star ArXiv
Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models arXiv Website - 3DV 2024

Single-view reconstruction

Title arXiv WebSite Github Pub.
Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior arXiv Website Star ICCV 2023
Zero-1-to-3: Zero-shot One Image to 3D Object arXiv Website Star ICCV 2023
Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model arXiv - Star ArXiv, 2023, 10
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image arXiv Website Star ArXiv, 2023, 09
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors arXiv Website Star ArXiv, 2023, 06
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation arXiv Website Star ArXiv

3D general editing

3D local editing

Title arXiv WebSite Github Pub.
FocalDreamer: Text-driven 3D Editing via Focal-fusion Assembly arXiv Website Star ArXiv, 2023, 08
DreamEditor: Text-Driven 3D Scene Editing with Neural Fields arXiv Website - SIGGRAPH ASIA 2023
OBJect 3DIT: Language-guided 3D-aware Image Editing arXiv Website - NeurIPS 2023
Vox-E: Text-guided Voxel Editing of 3D Objects arXiv Website Star ICCV 2023
Locally Stylized Neural Radiance Fields arXiv - - ICCV 2023
Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields arXiv Website Star ICCV 2023
Local 3D Editing via 3D Distillation of CLIP Knowledge arXiv - - CVPR 2023
RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models arXiv - - CVPR 2023
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field arXiv Website Star CVPR 2023
SKED: Sketch-guided Text-based 3D Editing arXiv - - CVPR 2023
Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision arXiv Website - CVPR 2023
Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts arXiv Website Star ArXiv
Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates arXiv - - ArXiv
ED-NeRF: Efficient Text-Guided Editing of 3D Scene using Latent Space NeRF arXiv - - ArXiv

3D (global) editing

Title arXiv WebSite Github Pub.
NeRF-Art: Text-Driven Neural Radiance Fields Stylization arXiv Website Star TVCG 2023
3D-aware Blending with Generative NeRFs arXiv Website Star ICCV 2023
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions arXiv Website Star ICCV 2023
Vox-E: Text-guided Voxel Editing of 3D Objects arXiv Website Star ICCV 2023
3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability arXiv - Star ICCV 2023
DreamBooth3D: Subject-Driven Text-to-3D Generation arXiv Website - ICCV 2023
Blending-NeRF: Text-Driven Localized Editing in Neural Radiance Fields arXiv Website - ICCV 2023
Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis arXiv Website - ICCV 2023
Edit-DiffNeRF: Editing 3D Neural Radiance Fields using 2D Diffusion Model arXiv Website - CVPR 2023
Frequency-Modulated Point Cloud Rendering with Easy Editing arXiv - Star CVPR 2023
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis arXiv - Star ArXiv, 2023, 09
Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields arXiv - - ArXiv, 2023, 06
Watch Your Steps: Local Image and Scene Editing by Text Instructions arXiv Website - ArXiv

Part-based editing

Title arXiv WebSite Github Pub.
PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision arXiv Website Star CVPR 2023
FocalDreamer: Text-driven 3D Editing via Focal-fusion Assembly arXiv Website Star ArXiv, 2023, 08
VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization arXiv Website - ArXiv

Inpainting

Title arXiv WebSite Github Pub.
OBJect 3DIT: Language-guided 3D-aware Image Editing arXiv Website - NeurIPS 2023
Reference-guided Controllable Inpainting of Neural Radiance Fields arXiv Website - ICCV 2023
Removing Objects From Neural Radiance Fields arXiv Website Star CVPR 2023
SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields arXiv Website Star CVPR 2023
InpaintNeRF360: Text-Guided 3D Inpainting on Unbounded Neural Radiance Fields arXiv - - ArXiv, 2023, 05
OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fields arXiv - Star ArXiv, 2023, 05
NeRF-In: Free-Form NeRF Inpainting with RGB-D Priors arXiv - - ArXiv, 2022, 06

Sketch-based editing

Title arXiv WebSite Github Pub.
FantASIA3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation arXiv Website Star ICCV 2023
Sketch2Mesh: Reconstructing and editing 3D shapes from sketches arXiv - Star ICCV 2021
SKED: Sketch-guided Text-based 3D Editing arXiv - - CVPR 2023
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures arXiv - Star ArXiv, 2022, 11

Reference-image based editing

Title arXiv WebSite Github Pub.
Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields arXiv Website Star ICCV 2023
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field arXiv Website Star CVPR 2023
Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates arXiv - - ArXiv

Appearance-specific editing

Stylization

Title arXiv WebSite Github Pub.
NeRF-Art: Text-Driven Neural Radiance Fields Stylization arXiv Website Star TVCG 2023
TeSTNeRF: Text-Driven 3D Style Transfer via Cross-Modal Learning arXiv - - IJCAI 2023
X-Mesh:Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance arXiv Website Star ICCV 2023
Locally Stylized Neural Radiance Fields arXiv - - ICCV 2023
Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylization arXiv - - CVPR 2023
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields arXiv Website Star CVPR 2023
StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields arXiv Website Star CVPR 2023
Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization arXiv Website Star CVPR 2023
MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field arXiv - - ArXiv
Text-driven Editing of 3D Scenes without Retraining arXiv Website Star ArXiv

Re-texture

Title arXiv WebSite Github Pub.
 ICE-NeRF: Interactive Color Editing of NeRFs via Decomposition-Aware Weight Optimization arXiv Website - ICCV 2023
Texture Generation on 3D Meshes with Point-UV Diffusion arXiv Website Star ICCV 2023
X-Mesh:Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance arXiv Website Star ICCV 2023
Text2Tex: Text-driven Texture Synthesis via Diffusion Models arXiv Website Star ICCV 2023
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing arXiv Website Star ECCV 2022
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields arXiv Website Star CVPR 2023
StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields arXiv Website Star CVPR 2023
Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization arXiv Website Star CVPR 2023
TensoIR: Tensorial Inverse Rendering arXiv Website Star CVPR 2023
Text-guided High-definition Consistency Texture Model arXiv - - ArXiv, 2023, 05
Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion arXiv Website - ArXiv, 2023, 04
ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context arXiv Website - ArXiv
Text-driven Editing of 3D Scenes without Retraining arXiv Website Star ArXiv
Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields arXiv Website - ArXiv
TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images arXiv Website - ArXiv
RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes arXiv Website Star ACM MM 2023
RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes arXiv Website Star ACM MM 2023

Re-light

Title arXiv WebSite Github Pub.
ENVIDR: Implicit Differentiable Renderer with Neural Environment Lighting arXiv Website Star ICCV 2023
ReNeRF: Relightable Neural Radiance Fields with Nearfield Lighting arXiv - - ICCV 2023
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis arXiv Website Star ICCV 2023
Neural Fields meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes arXiv - - CVPR 2023
TensoIR: Tensorial Inverse Rendering arXiv Website Star CVPR 2023

Deformation

Title arXiv WebSite Github Pub.
Real-time Locally Injective Volumetric Deformation arXiv - Star TOG 2021
Variational Harmonic Maps for Space Deformation arXiv - - TOG 2009
GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents arXiv Website - SIGGRAPH 2023
TextDeformer: Geometry Manipulation using Text Guidance arXiv Website Star SIGGRAPH 2023
Neural Shape Deformation Priors arXiv Website Star NeurIPS 2022
ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators arXiv - Star ICCV 2021
Deforming Radiance Fields with Cages arXiv Website Star ECCV 2022
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing arXiv Website Star ECCV 2022
NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds arXiv Website Star CVPR 2023
NeRF-Editing: Geometry Editing of Neural Radiance Fields arXiv - Star CVPR 2022
Neural Cages for Detail-Preserving 3D Deformations arXiv - Star CVPR 2020
SPIDR: SDF-based Neural Point Fields for Illumination and Deformation arXiv Website Star ArXiv, 2022, 10
VolTeMorph: Realtime, Controllable and Generalisable Animation of Volumetric Representations arXiv - - ArXiv, 2022, 08
MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Field arXiv Website Star ArXiv

Cage

Title arXiv WebSite Github Pub.
Neural Cages for Detail-Preserving 3D Deformations arXiv - Star CVPR 2020

Control point

Title arXiv WebSite Github Pub.
Real-time Locally Injective Volumetric Deformation arXiv - Star TOG 2021
Variational Harmonic Maps for Space Deformation arXiv - - TOG 2009
ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators arXiv - Star ICCV 2021
EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Points arXiv Website - CVPR 2023
NeRF-Editing: Geometry Editing of Neural Radiance Fields arXiv - Star CVPR 2022

Avatar-related editing

Title arXiv WebSite Github Pub.
AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars arXiv Website - SIGGRAPH ASIA 2023
Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization arXiv Website - SIGGRAPH 2023
CLIPFace: Text-guided Editing of Textured 3D Morphable Models arXiv Website Star SIGGRAPH 2023
DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance arXiv Website - SIGGRAPH 2023
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars arXiv Website Star SIGGRAPH 2022
FDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models arXiv - Star NeurIPS 2023
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields arXiv Website Star ICCV 2023
Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images arXiv - - ICCV 2023
FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fields arXiv Website - ICCV 2023
CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes arXiv Website Star ECCV 2022
Learning Locally Editable Virtual Humans arXiv Website Star CVPR 2023
MEGANE: Morphable Eyeglass and Avatar Network arXiv Website - CVPR 2023
UV Volumes for Real-time Rendering of Editable Free-view Human Performance arXiv Website Star CVPR 2023
HumanGen: Generating Human Radiance Fields with Explicit Priors arXiv - - CVPR 2023
TECA: Text-Guided Generation and Editing of Compositional 3D Avatars arXiv Website - ArXiv, 2023, 09
MagicAvatar: Multimodal Avatar Generation and Animation arXiv Website Star ArXiv, 2023, 08
HeadSculpt: Crafting 3D Head Avatars with Text arXiv Website - ArXiv, 2023, 06
Control4D: Dynamic Portrait Editing by Learning 4D GAN from 2D Diffusion-based Editor arXiv Website - ArXiv, 2023, 05
Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion arXiv Website - ArXiv, 2022, 12
LatentSwap3D: Semantic Edits on 3D Image GANs arXiv Website Star ArXiv
Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field arXiv - Star AAAI 2023
TADA! Text to Animatable Digital Avatars arXiv Website Star 3DV 2024

Scene editing

Title arXiv WebSite Github Pub.
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis arXiv Website Star ICCV 2023
Diffusion-based Generation, Optimization, and Planning in 3D Scenes arXiv Website - CVPR 2023
I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs arXiv Website Star CVPR 2023
Neural Fields meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes arXiv - - CVPR 2023
SIEDOB: Semantic Image Editing by Disentangling Object and Background arXiv - Star CVPR 2023
Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details arXiv - - CVPR 2023
NeRFEditor: Differentiable Style Decomposition for Full 3D Scene Editing arXiv Website - ArXiv 2022
Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes arXiv Website Star ArXiv

4D editing

Title arXiv WebSite Github Pub.
NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos arXiv - Star NeurIPS 2022
Text-guided High-definition Consistency Texture Model arXiv - - ArXiv, 2023, 05
Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields arXiv Website - ArXiv
Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image arXiv - - ACM MM 2023

About