Learning Structured Communication for Multi-agent Reinforcement Learning

  • 2020-02-11 07:19:45
  • Junjie Sheng, Xiangfeng Wang, Bo Jin, Junchi Yan, Wenhao Li, Tsung-Hui Chang, Jun Wang, Hongyuan Zha
  • 1

Abstract

This work explores the large-scale multi-agent communication mechanism undera multi-agent reinforcement learning (MARL) setting. We summarize the generalcategories of topology for communication structures in MARL literature, whichare often manually specified. Then we propose a novel framework termed asLearning Structured Communication (LSC) by using a more flexible and efficientcommunication topology. Our framework allows for adaptive agent grouping toform different hierarchical formations over episodes, which is generated by anauxiliary task combined with a hierarchical routing protocol. Given each formedtopology, a hierarchical graph neural network is learned to enable effectivemessage information generation and propagation among inter- and intra-groupcommunications. In contrast to existing communication mechanisms, our methodhas an explicit while learnable design for hierarchical communication.Experiments on challenging tasks show the proposed LSC enjoys highcommunication efficiency, scalability, and global cooperation capability.

 

Quick Read (beta)

loading the full paper ...