Abstract
Recently, deep multi-agent reinforcement learning (MARL) has shown thepromise to solve complex cooperative tasks. Its success is partly because ofparameter sharing among agents. However, such sharing may lead agents to behavesimilarly and limit their coordination capacity. In this paper, we aim tointroduce diversity in both optimization and representation of sharedmulti-agent reinforcement learning. Specifically, we propose aninformation-theoretical regularization to maximize the mutual informationbetween agents' identities and their trajectories, encouraging extensiveexploration and diverse individualized behaviors. In representation, weincorporate agent-specific modules in the shared neural network architecture,which are regularized by L1-norm to promote learning sharing among agents whilekeeping necessary diversity. Empirical results show that our method achievesstate-of-the-art performance on Google Research Football and super hardStarCraft II micromanagement tasks.