Abstract
Humans throw and catch objects all the time. However, such a seemingly commonskill introduces a lot of challenges for robots to achieve: The robots need tooperate such dynamic actions at high-speed, collaborate precisely, and interactwith diverse objects. In this paper, we design a system with two multi-fingerhands attached to robot arms to solve this problem. We train our system usingMulti-Agent Reinforcement Learning in simulation and perform Sim2Real transferto deploy on the real robots. To overcome the Sim2Real gap, we provide multiplenovel algorithm designs including learning a trajectory prediction model forthe object. Such a model can help the robot catcher has a real-time estimationof where the object will be heading, and then react accordingly. We conduct ourexperiments with multiple objects in the real-world system, and showsignificant improvements over multiple baselines. Our project page is availableat \url{https://binghao-huang.github.io/dynamic_handover/}.