Abstract
Cities, as the most fundamental environment of human life, encompass diversephysical elements such as buildings, roads and vegetation with complexinterconnection. Crafting realistic, interactive 3D urban environments plays acrucial role in constructing AI agents capable of perceiving, decision-making,and acting like humans in real-world environments. However, creatinghigh-fidelity 3D urban environments usually entails extensive manual labor fromdesigners, involving intricate detailing and accurate representation of complexurban features. Therefore, how to accomplish this in an automatical way remainsa longstanding challenge. Toward this problem, we propose UrbanWorld, the firstgenerative urban world model that can automatically create a customized,realistic and interactive 3D urban world with flexible control conditions.UrbanWorld incorporates four key stages in the automatical crafting pipeline:3D layout generation from openly accessible OSM data, urban scene planning anddesigning with a powerful urban multimodal large language model (Urban MLLM),controllable urban asset rendering with advanced 3D diffusion techniques, andfinally the MLLM-assisted scene refinement. The crafted high-fidelity 3D urbanenvironments enable realistic feedback and interactions for general AI andmachine perceptual systems in simulations. We are working on contributingUrbanWorld as an open-source and versatile platform for evaluating andimproving AI abilities in perception, decision-making, and interaction inrealistic urban environments.