Abstract
Neural Radiance Fields (NeRFs) have demonstrated remarkable proficiency insynthesizing photorealistic images of large-scale scenes. However, they areoften plagued by a loss of fine details and long rendering durations. 3DGaussian Splatting has recently been introduced as a potent alternative,achieving both high-fidelity visual results and accelerated renderingperformance. Nonetheless, scaling 3D Gaussian Splatting is fraught withchallenges. Specifically, large-scale scenes grapples with the integration ofobjects across multiple scales and disparate viewpoints, which often leads tocompromised efficacy as the Gaussians need to balance between detail levels.Furthermore, the generation of initialization points via COLMAP fromlarge-scale dataset is both computationally demanding and prone to incompletereconstructions. To address these challenges, we present Pyramidal 3D GaussianSplatting (PyGS) with NeRF Initialization. Our approach represent the scenewith a hierarchical assembly of Gaussians arranged in a pyramidal fashion. Thetop level of the pyramid is composed of a few large Gaussians, while eachsubsequent layer accommodates a denser collection of smaller Gaussians. Weeffectively initialize these pyramidal Gaussians through sampling a rapidlytrained grid-based NeRF at various frequencies. We group these pyramidalGaussians into clusters and use a compact weighting network to dynamicallydetermine the influence of each pyramid level of each cluster consideringcamera viewpoint during rendering. Our method achieves a significantperformance leap across multiple large-scale datasets and attains a renderingtime that is over 400 times faster than current state-of-the-art approaches.