Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels

  • 2021-04-01 17:30:45
  • Tze-Yang Tung, Szymon Kobus, Joan Roig Pujol, Deniz Gunduz
  • 0

Abstract

We propose a novel formulation of the "effectiveness problem" incommunications, put forth by Shannon and Weaver in their seminal work [2], byconsidering multiple agents communicating over a noisy channel in order toachieve better coordination and cooperation in a multi-agent reinforcementlearning (MARL) framework. Specifically, we consider a multi-agent partiallyobservable Markov decision process (MA-POMDP), in which the agents, in additionto interacting with the environment can also communicate with each other over anoisy communication channel. The noisy communication channel is consideredexplicitly as part of the dynamics of the environment and the message eachagent sends is part of the action that the agent can take. As a result, theagents learn not only to collaborate with each other but also to communicate"effectively" over a noisy channel. This framework generalizes both thetraditional communication problem, where the main goal is to convey a messagereliably over a noisy channel, and the "learning to communicate" framework thathas received recent attention in the MARL literature, where the underlyingcommunication channels are assumed to be error-free. We show via examples thatthe joint policy learned using the proposed framework is superior to that wherethe communication is considered separately from the underlying MA-POMDP. Thisis a very powerful framework, which has many real world applications, fromautonomous vehicle planning to drone swarm control, and opens up the richtoolbox of deep reinforcement learning for the design of multi-usercommunication systems.

 

Quick Read (beta)

loading the full paper ...