Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation

  • 2019-06-07 16:16:41
  • Ruohan Wang, Carlo Ciliberto, Pierluigi Amadori, Yiannis Demiris
  • 0

Abstract

We consider the problem of imitation learning from a finite set of experttrajectories, without access to reinforcement signals. The classical approachof extracting the expert's reward function via inverse reinforcement learning,followed by reinforcement learning is indirect and may be computationallyexpensive. Recent generative adversarial methods based on matching the policydistribution between the expert and the agent could be unstable duringtraining. We propose a new framework for imitation learning by estimating thesupport of the expert policy to compute a fixed reward function, which allowsus to re-frame imitation learning within the standard reinforcement learningsetting. We demonstrate the efficacy of our reward function on both discreteand continuous domains, achieving comparable or better performance than thestate of the art under different reinforcement learning algorithms.

 

Quick Read (beta)

loading the full paper ...