State-based Episodic Memory for Multi-Agent Reinforcement Learning

  • 2021-10-19 09:39:19
  • Xiao Ma, Wu-Jun Li
  • 5

Abstract

Multi-agent reinforcement learning (MARL) algorithms have made promisingprogress in recent years by leveraging the centralized training anddecentralized execution (CTDE) paradigm. However, existing MARL algorithmsstill suffer from the sample inefficiency problem. In this paper, we propose asimple yet effective approach, called state-based episodic memory (SEM), toimprove sample efficiency in MARL. SEM adopts episodic memory (EM) to supervisethe centralized training procedure of CTDE in MARL. To the best of ourknowledge, SEM is the first work to introduce EM into MARL. We cantheoretically prove that, when using for MARL, SEM has lower space complexityand time complexity than state and action based EM (SAEM), which is originallyproposed for single-agent reinforcement learning. Experimental results onStarCraft multi-agent challenge (SMAC) show that introducing episodic memoryinto MARL can improve sample efficiency and SEM can reduce storage cost andtime cost compared with SAEM.

 

Quick Read (beta)

loading the full paper ...