Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks

  • 2019-08-02 20:42:48
  • Roberto Martín-Martín, Michelle A. Lee, Rachel Gardner, Silvio Savarese, Jeannette Bohg, Animesh Garg
  • 0

Abstract

Reinforcement Learning (RL) of contact-rich manipulation tasks has yieldedimpressive results in recent years. While many studies in RL focus on varyingthe observation space or reward model, few efforts focused on the choice ofaction space (e.g. joint or end-effector space, position, velocity, etc.).However, studies in robot motion control indicate that choosing an action spacethat conforms to the characteristics of the task can simplify exploration andimprove robustness to disturbances. This paper studies the effect of differentaction spaces in deep RL and advocates for Variable Impedance Control inEnd-effector Space (VICES) as an advantageous action space for constrained andcontact-rich tasks. We evaluate multiple action spaces on three prototypicalmanipulation tasks: Path Following (task with no contact), Door Opening (taskwith kinematic constraints), and Surface Wiping (task with continuous contact).We show that VICES improves sample efficiency, maintains low energyconsumption, and ensures safety across all three experimental setups. Further,RL policies learned with VICES can transfer across different robot models insimulation, and from simulation to real for the same robot. Further informationis available at https://stanfordvl.github.io/vices.

 

Quick Read (beta)

loading the full paper ...