GaussianCity: Generative Gaussian Splatting for Unbounded 3D City Generation

  • 2024-06-10 18:59:55
  • Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, Ziwei Liu
  • 0

Abstract

3D city generation with NeRF-based methods shows promising generation resultsbut is computationally inefficient. Recently 3D Gaussian Splatting (3D-GS) hasemerged as a highly efficient alternative for object-level 3D generation.However, adapting 3D-GS from finite-scale 3D objects and humans toinfinite-scale 3D cities is non-trivial. Unbounded 3D city generation entailssignificant storage overhead (out-of-memory issues), arising from the need toexpand points to billions, often demanding hundreds of Gigabytes of VRAM for acity scene spanning 10km^2. In this paper, we propose GaussianCity, agenerative Gaussian Splatting framework dedicated to efficiently synthesizingunbounded 3D cities with a single feed-forward pass. Our key insights aretwo-fold: 1) Compact 3D Scene Representation: We introduce BEV-Point as ahighly compact intermediate representation, ensuring that the growth in VRAMusage for unbounded scenes remains constant, thus enabling unbounded citygeneration. 2) Spatial-aware Gaussian Attribute Decoder: We presentspatial-aware BEV-Point decoder to produce 3D Gaussian attributes, whichleverages Point Serializer to integrate the structural and contextualcharacteristics of BEV points. Extensive experiments demonstrate thatGaussianCity achieves state-of-the-art results in both drone-view andstreet-view 3D city generation. Notably, compared to CityDreamer, GaussianCityexhibits superior performance with a speedup of 60 times (10.72 FPS v.s. 0.18FPS).

 

Quick Read (beta)

loading the full paper ...