Abstract
Envisioning physically plausible outcomes from a single image requires a deepunderstanding of the world's dynamics. To address this, we introduce PhysGen3D,a novel framework that transforms a single image into an amodal,camera-centric, interactive 3D scene. By combining advanced image-basedgeometric and semantic understanding with physics-based simulation, PhysGen3Dcreates an interactive 3D world from a static image, enabling us to "imagine"and simulate future scenarios based on user input. At its core, PhysGen3Destimates 3D shapes, poses, physical and lighting properties of objects,thereby capturing essential physical attributes that drive realistic objectinteractions. This framework allows users to specify precise initialconditions, such as object speed or material properties, for enhanced controlover generated video outcomes. We evaluate PhysGen3D's performance againstclosed-source state-of-the-art (SOTA) image-to-video models, including Pika,Kling, and Gen-3, showing PhysGen3D's capacity to generate videos withrealistic physics while offering greater flexibility and fine-grained control.Our results show that PhysGen3D achieves a unique balance of photorealism,physical plausibility, and user-driven interactivity, opening new possibilitiesfor generating dynamic, physics-grounded video from an image.