Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learning

  • 2022-05-11 04:12:49
  • Shuhan Qi, Shuhao Zhang, Xiaohan Hou, Jiajia Zhang, Xuan Wang, Jing Xiao
  • 1

Abstract

Multi-agent reinforcement learning for incomplete information environmentshas attracted extensive attention from researchers. However, due to the slowsample collection and poor sample exploration, there are still some problems inmulti-agent reinforcement learning, such as unstable model iteration and lowtraining efficiency. Moreover, most of the existing distributed framework areproposed for single-agent reinforcement learning and not suitable formulti-agent. In this paper, we design an distributed MARL framework based onthe actor-work-learner architecture. In this framework, multiple asynchronousenvironment interaction modules can be deployed simultaneously, which greatlyimproves the sample collection speed and sample diversity. Meanwhile, to makefull use of computing resources, we decouple the model iteration fromenvironment interaction, and thus accelerate the policy iteration. Finally, weverified the effectiveness of propose framework in MaCA military simulationenvironment and the SMAC 3D realtime strategy gaming environment withimcomplete information characteristics.

 

Quick Read (beta)

loading the full paper ...