Intrinsically Motivated Self-supervised Learning in Reinforcement Learning

  • 2022-05-13 10:02:22
  • Yue Zhao, Chenzhuang Du, Hang Zhao, Tiejun Li
  • 0

Abstract

In vision-based reinforcement learning (RL) tasks, it is prevalent to assignauxiliary tasks with a surrogate self-supervised loss so as to obtain moresemantic representations and improve sample efficiency. However, abundantinformation in self-supervised auxiliary tasks has been disregarded, since therepresentation learning part and the decision-making part are separated. Tosufficiently utilize information in auxiliary tasks, we present a simple yeteffective idea to employ self-supervised loss as an intrinsic reward, calledIntrinsically Motivated Self-Supervised learning in Reinforcement learning(IM-SSR). We formally show that the self-supervised loss can be decomposed asexploration for novel states and robustness improvement from nuisanceelimination. IM-SSR can be effortlessly plugged into any reinforcement learningwith self-supervised auxiliary objectives with nearly no additional cost.Combined with IM-SSR, the previous underlying algorithms achieve salientimprovements on both sample efficiency and generalization in variousvision-based robotics tasks from the DeepMind Control Suite, especially whenthe reward signal is sparse.

 

Quick Read (beta)

loading the full paper ...