HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism

  • 2021-10-14 10:43:47
  • Zhiwei Xu, Yunpeng Bai, Bin Zhang, Dapeng Li, Guoliang Fan
  • 0

Abstract

Multi-agent reinforcement learning often suffers from the exponentiallylarger action space caused by a large number of agents. In this paper, wepropose a novel value decomposition framework HAVEN based on hierarchicalreinforcement learning for the fully cooperative multi-agent problems. In orderto address instabilities that arise from the concurrent optimization ofhigh-level and low-level policies and another concurrent optimization ofagents, we introduce the dual coordination mechanism of inter-layer strategiesand inter-agent strategies. HAVEN does not require domain knowledge andpretraining at all, and can be applied to any value decomposition variants. Ourmethod is demonstrated to achieve superior results to many baselines onStarCraft II micromanagement tasks and offers an efficient solution tomulti-agent hierarchical reinforcement learning in fully cooperative scenarios.

 

Quick Read (beta)

loading the full paper ...