Abstract
Cooperative Multi-Agent Reinforcement Learning (MARL) necessitates seamlesscollaboration among agents, often represented by an underlying relation graph.Existing methods for learning this graph primarily focus on agent-pairrelations, neglecting higher-order relationships. While several approachesattempt to extend cooperation modelling to encompass behaviour similaritieswithin groups, they commonly fall short in concurrently learning the latentgraph, thereby constraining the information exchange among partially observedagents. To overcome these limitations, we present a novel approach to infer theGroup-Aware Coordination Graph (GACG), which is designed to capture both thecooperation between agent pairs based on current observations and group-leveldependencies from behaviour patterns observed across trajectories. This graphis further used in graph convolution for information exchange between agentsduring decision-making. To further ensure behavioural consistency among agentswithin the same group, we introduce a group distance loss, which promotes groupcohesion and encourages specialization between groups. Our evaluations,conducted on StarCraft II micromanagement tasks, demonstrate GACG's superiorperformance. An ablation study further provides experimental evidence of theeffectiveness of each component of our method.