Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting

  • 2024-07-25 18:15:58
  • Jeongmin Bae, Seoha Kim, Youngsik Yun, Hahyun Lee, Gun Bang, Youngjung Uh
  • 0

Abstract

As 3D Gaussian Splatting (3DGS) provides fast and high-quality novel viewsynthesis, it is a natural extension to deform a canonical 3DGS to multipleframes for representing a dynamic scene. However, previous works fail toaccurately reconstruct complex dynamic scenes. We attribute the failure to thedesign of the deformation field, which is built as a coordinate-based function.This approach is problematic because 3DGS is a mixture of multiple fieldscentered at the Gaussians, not just a single coordinate-based framework. Toresolve this problem, we define the deformation as a function of per-Gaussianembeddings and temporal embeddings. Moreover, we decompose deformations ascoarse and fine deformations to model slow and fast movements, respectively.Also, we introduce a local smoothness regularization for per-Gaussian embeddingto improve the details in dynamic regions. Project page:https://jeongminb.github.io/e-d3dgs/

 

Quick Read (beta)

loading the full paper ...