Abstract
Large-scale and diverse datasets are vital for training robust roboticmanipulation policies, yet existing data collection methods struggle to balancescale, diversity, and quality. Simulation offers scalability but suffers fromsim-to-real gaps, while teleoperation yields high-quality demonstrations withlimited diversity and high labor cost. We introduce FieldGen, a field-guideddata generation framework that enables scalable, diverse, and high-qualityreal-world data collection with minimal human supervision. FieldGen decomposesmanipulation into two stages: a pre-manipulation phase, allowing trajectorydiversity, and a fine manipulation phase requiring expert precision. Humandemonstrations capture key contact and pose information, after which anattraction field automatically generates diverse trajectories converging tosuccessful configurations. This decoupled design combines scalable trajectorydiversity with precise supervision. Moreover, FieldGen-Reward augmentsgenerated data with reward annotations to further enhance policy learning.Experiments demonstrate that policies trained with FieldGen achieve highersuccess rates and improved stability compared to teleoperation-based baselines,while significantly reducing human effort in long-term real-world datacollection. Webpage is available at https://fieldgen.github.io/.