Abstract
3D Gaussian Splatting techniques have enabled efficient photo-realisticrendering of static scenes. Recent works have extended these approaches tosupport surface reconstruction and tracking. However, tracking dynamic surfaceswith 3D Gaussians remains challenging due to complex topology changes, such assurfaces appearing, disappearing, or splitting. To address these challenges, wepropose GSTAR, a novel method that achieves photo-realistic rendering, accuratesurface reconstruction, and reliable 3D tracking for general dynamic sceneswith changing topology. Given multi-view captures as input, GSTAR bindsGaussians to mesh faces to represent dynamic objects. For surfaces withconsistent topology, GSTAR maintains the mesh topology and tracks the meshesusing Gaussians. In regions where topology changes, GSTAR adaptively unbindsGaussians from the mesh, enabling accurate registration and the generation ofnew surfaces based on these optimized Gaussians. Additionally, we introduce asurface-based scene flow method that provides robust initialization fortracking between frames. Experiments demonstrate that our method effectivelytracks and reconstructs dynamic surfaces, enabling a range of applications. Ourproject page with the code release is available athttps://eth-ait.github.io/GSTAR/.