Abstract
Mean field theory provides an effective way of scaling multiagentreinforcement learning algorithms to environments with many agents that can beabstracted by a virtual mean agent. In this paper, we extend mean fieldmultiagent algorithms to multiple types. The types enable the relaxation of acore assumption in mean field reinforcement learning, which is that all agentsin the environment are playing almost similar strategies and have the samegoal. We conduct experiments on three different testbeds for the field of manyagent reinforcement learning, based on the standard MAgents framework. Weconsider two different kinds of mean field environments: a) Games where agentsbelong to predefined types that are known a priori and b) Games where the typeof each agent is unknown and therefore must be learned based on observations.We introduce new algorithms for each type of game and demonstrate theirsuperior performance over state of the art algorithms that assume that allagents belong to the same type and other baseline algorithms in the MAgentframework.