Abstract
Existing multi-agent reinforcement learning methods are limited typically toa small number of agents. When the agent number increases largely, the learningbecomes intractable due to the curse of the dimensionality and the exponentialgrowth of user interactions. In this paper, we present Mean Field ReinforcementLearning where the interactions within the population of agents areapproximated by those between a single agent and the average effect from theoverall population or neighboring agents; the interplay between the twoentities is mutually reinforced: the learning of the individual agent's optimalpolicy depends on the dynamics of the population, while the dynamics of thepopulation change according to the collective patterns of the individualpolicies. We develop practical mean field Q-learning and mean fieldActor-Critic algorithms and analyze the convergence of the solution.Experiments on resource allocation, Ising model estimation, and battle gametasks verify the learning effectiveness of our mean field approaches inhandling many-agent interactions in population.