Abstract
We present a low-cost data generation pipeline that integrates physics-basedsimulation, human demonstrations, and model-based planning to efficientlygenerate large-scale, high-quality datasets for contact-rich roboticmanipulation tasks. Starting with a small number of embodiment-flexible humandemonstrations collected in a virtual reality simulation environment, thepipeline refines these demonstrations using optimization-based kinematicretargeting and trajectory optimization to adapt them across various robotembodiments and physical parameters. This process yields a diverse, physicallyconsistent dataset that enables cross-embodiment data transfer, and offers thepotential to reuse legacy datasets collected under different hardwareconfigurations or physical parameters. We validate the pipeline's effectivenessby training diffusion policies from the generated datasets for challengingcontact-rich manipulation tasks across multiple robot embodiments, including afloating Allegro hand and bimanual robot arms. The trained policies aredeployed zero-shot on hardware for bimanual iiwa arms, achieving high successrates with minimal human input. Project website:https://lujieyang.github.io/physicsgen/.