Abstract
Deep reinforcement learning has been applied successfully to solve variousreal-world problems and the number of its applications in the multi-agentsettings has been increasing. Multi-agent learning distinctly poses significantchallenges in the effort to allocate a concealed communication medium. Agentsreceive thorough knowledge from the medium to determine subsequent actions in adistributed nature. Apparently, the goal is to leverage the cooperation ofmultiple agents to achieve a designated objective efficiently. Recent studiestypically combine a specialized neural network with reinforcement learning toenable communication between agents. This approach, however, limits the numberof agents or necessitates the homogeneity of the system. In this paper, we haveproposed a more scalable approach that not only deals with a great number ofagents but also enables collaboration between dissimilar functional agents andcompatibly combined with any deep reinforcement learning methods. Specifically,we create a global communication map to represent the status of each agent inthe system visually. The visual map and the environmental state are fed to ashared-parameter network to train multiple agents concurrently. Finally, weselect the Asynchronous Advantage Actor-Critic (A3C) algorithm to demonstrateour proposed scheme, namely Visual communication map for Multi-agent A3C(VMA3C). Simulation results show that the use of visual communication mapimproves the performance of A3C regarding learning speed, reward achievement,and robustness in multi-agent problems.