Abstract
In the realm of robot-assisted minimally invasive surgery, dynamic scenereconstruction can significantly enhance downstream tasks and improve surgicaloutcomes. Neural Radiance Fields (NeRF)-based methods have recently risen toprominence for their exceptional ability to reconstruct scenes. Nonetheless,these methods are hampered by slow inference, prolonged training, andsubstantial computational demands. Additionally, some rely on stereo depthestimation, which is often infeasible due to the high costs and logisticalchallenges associated with stereo cameras. Moreover, the monocularreconstruction quality for deformable scenes is currently inadequate. Toovercome these obstacles, we present Endo-4DGS, an innovative, real-timeendoscopic dynamic reconstruction approach that utilizes 4D Gaussian Splatting(GS) and requires no ground truth depth data. This method extends 3D GS byincorporating a temporal component and leverages a lightweight MLP to capturetemporal Gaussian deformations. This effectively facilitates the reconstructionof dynamic surgical scenes with variable conditions. We also integrateDepth-Anything to generate pseudo-depth maps from monocular views, enhancingthe depth-guided reconstruction process. Our approach has been validated on twosurgical datasets, where it has proven to render in real-time, computeefficiently, and reconstruct with remarkable accuracy. These results underlinethe vast potential of Endo-4DGS to improve surgical assistance.