Abstract
Achieving realistic simulations of humans interacting with a wide range ofobjects has long been a fundamental goal. Extending physics-based motionimitation to complex human-object interactions (HOIs) is challenging due tointricate human-object coupling, variability in object geometries, andartifacts in motion capture data, such as inaccurate contacts and limited handdetail. We introduce InterMimic, a framework that enables a single policy torobustly learn from hours of imperfect MoCap data covering diverse full-bodyinteractions with dynamic and varied objects. Our key insight is to employ acurriculum strategy -- perfect first, then scale up. We first trainsubject-specific teacher policies to mimic, retarget, and refine motion capturedata. Next, we distill these teachers into a student policy, with the teachersacting as online experts providing direct supervision, as well as high-qualityreferences. Notably, we incorporate RL fine-tuning on the student policy tosurpass mere demonstration replication and achieve higher-quality solutions.Our experiments demonstrate that InterMimic produces realistic and diverseinteractions across multiple HOI datasets. The learned policy generalizes in azero-shot manner and seamlessly integrates with kinematic generators, elevatingthe framework from mere imitation to generative modeling of complexhuman-object interactions.