Abstract
This paper introduces Honor of Kings Arena, a reinforcement learning (RL)environment based on Honor of Kings, one of the world's most popular games atpresent. Compared to other environments studied in most previous work, ourspresents new generalization challenges for competitive reinforcement learning.It is a multi-agent problem with one agent competing against its opponent; andit requires the generalization ability as it has diverse targets to control anddiverse opponents to compete with. We describe the observation, action, andreward specifications for the Honor of Kings domain and provide an open-sourcePython-based interface for communicating with the game engine. We providetwenty target heroes with a variety of tasks in Honor of Kings Arena andpresent initial baseline results for RL-based methods with feasible computingresources. Finally, we showcase the generalization challenges imposed by Honorof Kings Arena and possible remedies to the challenges. All of the software,including the environment-class, are publicly available athttps://github.com/tencent-ailab/hok_env . The documentation is available athttps://aiarena.tencent.com/hok/doc/ .