Object-centric Forward Modeling for Model Predictive Control

  • 2019-10-08 17:58:10
  • Yufei Ye, Dhiraj Gandhi, Abhinav Gupta, Shubham Tulsiani
  • 0

Abstract

We present an approach to learn an object-centric forward model, and showthat this allows us to plan for sequences of actions to achieve distant desiredgoals. We propose to model a scene as a collection of objects, each with anexplicit spatial location and implicit visual feature, and learn to model theeffects of actions using random interaction data. Our model allows capturingthe robot-object and object-object interactions, and leads to moresample-efficient and accurate predictions. We show that this learned model canbe leveraged to search for action sequences that lead to desired goalconfigurations, and that in conjunction with a learned correction module, thisallows for robust closed loop execution. We present experiments both insimulation and the real world, and show that our approach improves overalternate implicit or pixel-space forward models. Please see our project page(https://judyye.github.io/ocmpc/) for result videos.

 

Quick Read (beta)

loading the full paper ...