Abstract
We use reinforcement learning (RL) to learn dexterous in-hand manipulationpolicies which can perform vision-based object reorientation on a physicalShadow Dexterous Hand. The training is performed in a simulated environment inwhich we randomize many of the physical properties of the system like frictioncoefficients and an object's appearance. Our policies transfer to the physicalrobot despite being trained entirely in simulation. Our method does not rely onany human demonstrations, but many behaviors found in human manipulation emergenaturally, including finger gaiting, multi-finger coordination, and thecontrolled use of gravity. Our results were obtained using the same distributedRL system that was used to train OpenAI Five. We also include a video of ourresults: https://youtu.be/jwSbzNHGflM.