Abstract
Recent advancements in 3D Gaussian Splatting (3DGS) and Neural RadianceFields (NeRF) have achieved impressive results in real-time 3D reconstructionand novel view synthesis. However, these methods struggle in large-scale,unconstrained environments where sparse and uneven input coverage, transientocclusions, appearance variability, and inconsistent camera settings lead todegraded quality. We propose GS-Diff, a novel 3DGS framework guided by amulti-view diffusion model to address these limitations. By generatingpseudo-observations conditioned on multi-view inputs, our method transformsunder-constrained 3D reconstruction problems into well-posed ones, enablingrobust optimization even with sparse data. GS-Diff further integrates severalenhancements, including appearance embedding, monocular depth priors, dynamicobject modeling, anisotropy regularization, and advanced rasterizationtechniques, to tackle geometric and photometric challenges in real-worldsettings. Experiments on four benchmarks demonstrate that GS-Diff consistentlyoutperforms state-of-the-art baselines by significant margins.