Abstract
Graphical user interface (UI) software has undergone a fundamentaltransformation from traditional two-dimensional (2D) desktop/web/mobileinterfaces to spatial three-dimensional (3D) environments. While existing workhas made remarkable success in automated 2D software generation, such asHTML/CSS and mobile app interface code synthesis, the generation of 3D softwarestill remains under-explored. Current methods for 3D software generationusually generate the 3D environments as a whole and cannot modify or controlspecific elements in the software. Furthermore, these methods struggle tohandle the complex spatial and semantic constraints inherent in the real world.To address the challenges, we present Scenethesis, a novelrequirement-sensitive 3D software synthesis approach that maintains formaltraceability between user specifications and generated 3D software. Scenethesisis built upon ScenethesisLang, a domain-specific language that serves as agranular constraint-aware intermediate representation (IR) to bridge naturallanguage requirements and executable 3D software. It serves both as acomprehensive scene description language enabling fine-grained modification of3D software elements and as a formal constraint-expressive specificationlanguage capable of expressing complex spatial constraints. By decomposing 3Dsoftware synthesis into stages operating on ScenethesisLang, Scenethesisenables independent verification, targeted modification, and systematicconstraint satisfaction. Our evaluation demonstrates that Scenethesisaccurately captures over 80% of user requirements and satisfies more than 90%of hard constraints while handling over 100 constraints simultaneously.Furthermore, Scenethesis achieves a 42.8% improvement in BLIP-2 visualevaluation scores compared to the state-of-the-art method.