Abstract
Swarm systems constitute a challenging problem for reinforcement learning(RL) as the algorithm needs to learn decentralized control policies that cancope with limited local sensing and communication abilities of the agents.While it is often difficult to directly define the behavior of the agents,simple communication protocols can be defined more easily using prior knowledgeabout the given task. In this paper, we propose a number of simplecommunication protocols that can be exploited by deep reinforcement learning tofind decentralized control policies in a multi-robot swarm environment. Theprotocols are based on histograms that encode the local neighborhood relationsof the agents and can also transmit task-specific information, such as theshortest distance and direction to a desired target. In our framework, we usean adaptation of Trust Region Policy Optimization to learn complexcollaborative tasks, such as formation building and building a communicationlink. We evaluate our findings in a simulated 2D-physics environment, andcompare the implications of different communication protocols.