Abstract
3D Gaussian Splatting has emerged as an alternative 3D representation ofNeural Radiance Fields (NeRFs), benefiting from its high-quality renderingresults and real-time rendering speed. Considering the 3D Gaussianrepresentation remains unparsed, it is necessary first to execute objectsegmentation within this domain. Subsequently, scene editing and collisiondetection can be performed, proving vital to a multitude of applications, suchas virtual reality (VR), augmented reality (AR), game/movie production, etc. Inthis paper, we propose a novel approach to achieve object segmentation in 3DGaussian via an interactive procedure without any training process and learnedparameters. We refer to the proposed method as SA-GS, for Segment Anything in3D Gaussians. Given a set of clicked points in a single input view, SA-GS cangeneralize SAM to achieve 3D consistent segmentation via the proposedmulti-view mask generation and view-wise label assignment methods. We alsopropose a cross-view label-voting approach to assign labels from differentviews. In addition, in order to address the boundary roughness issue ofsegmented objects resulting from the non-negligible spatial sizes of 3DGaussian located at the boundary, SA-GS incorporates the simple but effectiveGaussian Decomposition scheme. Extensive experiments demonstrate that SA-GSachieves high-quality 3D segmentation results, which can also be easily appliedfor scene editing and collision detection tasks. Codes will be released soon.