Cooperative Exploration for Multi-Agent Deep Reinforcement Learning

  • 2021-07-23 20:06:32
  • Iou-Jen Liu, Unnat Jain, Raymond A. Yeh, Alexander G. Schwing
  • 1


Exploration is critical for good results in deep reinforcement learning andhas attracted much attention. However, existing multi-agent deep reinforcementlearning algorithms still use mostly noise-based techniques. Very recently,exploration methods that consider cooperation among multiple agents have beendeveloped. However, existing methods suffer from a common challenge: agentsstruggle to identify states that are worth exploring, and hardly coordinateexploration efforts toward those states. To address this shortcoming, in thispaper, we propose cooperative multi-agent exploration (CMAE): agents share acommon goal while exploring. The goal is selected from multiple projected statespaces via a normalized entropy-based technique. Then, agents are trained toreach this goal in a coordinated manner. We demonstrate that CMAE consistentlyoutperforms baselines on various tasks, including a sparse-reward version ofthe multiple-particle environment (MPE) and the Starcraft multi-agent challenge(SMAC).


Quick Read (beta)

loading the full paper ...