Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning

  • 2018-04-16 03:39:11
  • Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, Thomas Funkhouser
  • 0

Abstract

Skilled robotic manipulation benefits from complex synergies betweennon-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushingcan help rearrange cluttered objects to make space for arms and fingers;likewise, grasping can help displace objects to make pushing movements moreprecise and collision-free. In this work, we demonstrate that it is possible todiscover and learn these synergies from scratch through model-free deepreinforcement learning. Our method involves training two fully convolutionalnetworks that map from visual observations to actions: one infers the utilityof pushes for a dense pixel-wise sampling of end effector orientations andlocations, while the other does the same for grasping. Both networks aretrained jointly in a Q-learning framework and are entirely self-supervised bytrial and error, where rewards are provided from successful grasps. In thisway, our policy learns pushing motions that enable future grasps, whilelearning grasps that can leverage past pushes. During picking experiments inboth simulation and real-world scenarios, we find that our system quicklylearns complex behaviors amid challenging cases of clutter, and achieves bettergrasping success rates and picking efficiencies than baseline alternativesafter only a few hours of training. We further demonstrate that our method iscapable of generalizing to novel objects. Qualitative results (videos), code,pre-trained models, and simulation environments are available athttp://vpg.cs.princeton.edu

 

Quick Read (beta)

loading the full paper ...