Representation Learning for Out-Of-Distribution Generalization in Reinforcement Learning

  • 2021-07-12 18:49:48
  • Andrea Dittadi, Frederik Träuble, Manuel Wüthrich, Felix Widmaier, Peter Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, Stefan Bauer
  • 1

Abstract

Learning data representations that are useful for various downstream tasks isa cornerstone of artificial intelligence. While existing methods are typicallyevaluated on downstream tasks such as classification or generative imagequality, we propose to assess representations through their usefulness indownstream control tasks, such as reaching or pushing objects. By training over10,000 reinforcement learning policies, we extensively evaluate to what extentdifferent representation properties affect out-of-distribution (OOD)generalization. Finally, we demonstrate zero-shot transfer of these policiesfrom simulation to the real world, without any domain randomization orfine-tuning. This paper aims to establish the first systematic characterizationof the usefulness of learned representations for real-world OOD downstreamtasks.

 

Quick Read (beta)

loading the full paper ...