Abstract
We study the problem of functional retargeting: learning dexterousmanipulation policies to track object states from human hand-objectdemonstrations. We focus on long-horizon, bimanual tasks with articulatedobjects, which is challenging due to large action space, spatiotemporaldiscontinuities, and embodiment gap between human and robot hands. We proposeDexMachina, a novel curriculum-based algorithm: the key idea is to use virtualobject controllers with decaying strength: an object is first drivenautomatically towards its target states, such that the policy can graduallylearn to take over under motion and contact guidance. We release a simulationbenchmark with a diverse set of tasks and dexterous hands, and show thatDexMachina significantly outperforms baseline methods. Our algorithm andbenchmark enable a functional comparison for hardware designs, and we presentkey findings informed by quantitative and qualitative results. With the recentsurge in dexterous hand development, we hope this work will provide a usefulplatform for identifying desirable hardware capabilities and lower the barrierfor contributing to future research. Videos and more athttps://project-dexmachina.github.io/