Abstract
We introduce HouseCrafter, a novel approach that can lift a floorplan into acomplete large 3D indoor scene (e.g., a house). Our key insight is to adapt a2D diffusion model, which is trained on web-scale images, to generateconsistent multi-view color (RGB) and depth (D) images across differentlocations of the scene. Specifically, the RGB-D images are generatedautoregressively in a batch-wise manner along sampled locations based on thefloorplan, where previously generated images are used as condition to thediffusion model to produce images at nearby locations. The global floorplan andattention design in the diffusion model ensures the consistency of thegenerated images, from which a 3D scene can be reconstructed. Through extensiveevaluation on the 3D-Front dataset, we demonstrate that HouseCraft can generatehigh-quality house-scale 3D scenes. Ablation studies also validate theeffectiveness of different design choices. We will release our code and modelweights. Project page: https://neu-vi.github.io/houseCrafter/