Abstract
Depth maps are widely used in feed-forward 3D Gaussian Splatting (3DGS)pipelines by unprojecting them into 3D point clouds for novel view synthesis.This approach offers advantages such as efficient training, the use of knowncamera poses, and accurate geometry estimation. However, depth discontinuitiesat object boundaries often lead to fragmented or sparse point clouds, degradingrendering quality -- a well-known limitation of depth-based representations. Totackle this issue, we introduce PM-Loss, a novel regularization loss based on apointmap predicted by a pre-trained transformer. Although the pointmap itselfmay be less accurate than the depth map, it effectively enforces geometricsmoothness, especially around object boundaries. With the improved depth map,our method significantly improves the feed-forward 3DGS across variousarchitectures and scenes, delivering consistently better rendering results. Ourproject page: https://aim-uofa.github.io/PMLoss