Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning

  • 2020-03-29 17:42:00
  • Yongyuan Liang, Bangwei Li
  • 0

Abstract

Multi-agent reinforcement learning is a standard framework for modelingmulti-agent interactions applied in real-world scenarios. Inspired byexperience sharing in human groups, learning knowledge parallel reusing betweenagents can potentially promote team learning performance, especially inmulti-task environments. When all agents interact with the environment andlearn simultaneously, how each independent agent selectively learns from otheragents' behavior knowledge is a problem that we need to solve. This paperproposes a novel knowledge transfer framework in MARL, PAT (ParallelAttentional Transfer). We design two acting modes in PAT, student mode andself-learning mode. Each agent in our approach trains a decentralized studentactor-critic to determine its acting mode at each time step. When agents areunfamiliar with the environment, the shared attention mechanism in student modeeffectively selects learning knowledge from other agents to decide agents'actions. PAT outperforms state-of-the-art empirical evaluation results againstthe prior advising approaches. Our approach not only significantly improvesteam learning rate and global performance, but also is flexible andtransferable to be applied in various multi-agent systems.

 

Quick Read (beta)

loading the full paper ...