Abstract
Mimicry is a fundamental learning mechanism in humans, enabling individualsto learn new tasks by observing and imitating experts. However, applying thisability to robots presents significant challenges due to the inherentdifferences between human and robot embodiments in both their visual appearanceand physical capabilities. While previous methods bridge this gap usingcross-embodiment datasets with shared scenes and tasks, collecting such aligneddata between humans and robots at scale is not trivial. In this paper, wepropose UniSkill, a novel framework that learns embodiment-agnostic skillrepresentations from large-scale cross-embodiment video data without anylabels, enabling skills extracted from human video prompts to effectivelytransfer to robot policies trained only on robot data. Our experiments in bothsimulation and real-world environments show that our cross-embodiment skillssuccessfully guide robots in selecting appropriate actions, even with unseenvideo prompts. The project website can be found at:https://kimhanjung.github.io/UniSkill.