Abstract
Reinforcement learning has delivered promising results in achieving human- oreven superhuman-level capabilities across diverse problem domains, but successin dexterous robot manipulation remains limited. This work investigates the keychallenges in applying reinforcement learning to solve a collection ofcontact-rich manipulation tasks on a humanoid embodiment. We introduce noveltechniques to overcome the identified challenges with empirical validation. Ourmain contributions include an automated real-to-sim tuning module that bringsthe simulated environment closer to the real world, a generalized reward designscheme that simplifies reward engineering for long-horizon contact-richmanipulation tasks, a divide-and-conquer distillation process that improves thesample efficiency of hard-exploration problems while maintaining sim-to-realperformance, and a mixture of sparse and dense object representations to bridgethe sim-to-real perception gap. We show promising results on three humanoiddexterous manipulation tasks, with ablation studies on each technique. Our workpresents a successful approach to learning humanoid dexterous manipulationusing sim-to-real reinforcement learning, achieving robust generalization andhigh performance without the need for human demonstration.