Abstract
We present a method that simultaneously addresses the tasks of dynamic scenenovel-view synthesis and six degree-of-freedom (6-DOF) tracking of all densescene elements. We follow an analysis-by-synthesis framework, inspired byrecent work that models scenes as a collection of 3D Gaussians which areoptimized to reconstruct input images via differentiable rendering. To modeldynamic scenes, we allow Gaussians to move and rotate over time while enforcingthat they have persistent color, opacity, and size. By regularizing Gaussians'motion and rotation with local-rigidity constraints, we show that our Dynamic3D Gaussians correctly model the same area of physical space over time,including the rotation of that space. Dense 6-DOF tracking and dynamicreconstruction emerges naturally from persistent dynamic view synthesis,without requiring any correspondence or flow as input. We demonstrate a largenumber of downstream applications enabled by our representation, includingfirst-person view synthesis, dynamic compositional scene synthesis, and 4Dvideo editing.