Abstract
State-of-the-art 3D-aware generative models rely on coordinate-based MLPs toparameterize 3D radiance fields. While demonstrating impressive results,querying an MLP for every sample along each ray leads to slow rendering.Therefore, existing approaches often render low-resolution feature maps andprocess them with an upsampling network to obtain the final image. Albeitefficient, neural rendering often entangles viewpoint and content such thatchanging the camera pose results in unwanted changes of geometry or appearance.Motivated by recent results in voxel-based novel view synthesis, we investigatethe utility of sparse voxel grid representations for fast and 3D-consistentgenerative modeling in this paper. Our results demonstrate that monolithic MLPscan indeed be replaced by 3D convolutions when combining sparse voxel gridswith progressive growing, free space pruning and appropriate regularization. Toobtain a compact representation of the scene and allow for scaling to highervoxel resolutions, our model disentangles the foreground object (modeled in 3D)from the background (modeled in 2D). In contrast to existing approaches, ourmethod requires only a single forward pass to generate a full 3D scene. Ithence allows for efficient rendering from arbitrary viewpoints while yielding3D consistent results with high visual fidelity.