Abstract
In this work, we present Fed3DGS, a scalable 3D reconstruction frameworkbased on 3D Gaussian splatting (3DGS) with federated learning. Existingcity-scale reconstruction methods typically adopt a centralized approach, whichgathers all data in a central server and reconstructs scenes. The approachhampers scalability because it places a heavy load on the server and demandsextensive data storage when reconstructing scenes on a scale beyond city-scale.In pursuit of a more scalable 3D reconstruction, we propose a federatedlearning framework with 3DGS, which is a decentralized framework and canpotentially use distributed computational resources across millions of clients.We tailor a distillation-based model update scheme for 3DGS and introduceappearance modeling for handling non-IID data in the scenario of 3Dreconstruction with federated learning. We simulate our method on severallarge-scale benchmarks, and our method demonstrates rendered image qualitycomparable to centralized approaches. In addition, we also simulate our methodwith data collected in different seasons, demonstrating that our framework canreflect changes in the scenes and our appearance modeling captures changes dueto seasonal variations.