Abstract
In this paper, we propose Rogue-Gym, a simple and classic style roguelikegame built for evaluating generalization in reinforcement learning (RL).Combined with the recent progress of deep neural networks, RL has successfullytrained human-level agents without human knowledge in many games such as thosefor Atari 2600. However, it has been pointed out that agents trained with RLmethods often overfit the training environment, and they work poorly inslightly different environments. To investigate this problem, some researchenvironments with procedural content generation have been proposed. Followingthese studies, we propose the use of roguelikes as a benchmark for evaluatingthe generalization ability of RL agents. In our Rogue-Gym, agents need toexplore dungeons that are structured differently each time they start a newgame. Thanks to the very diverse structures of the dungeons, we believe thatthe generalization benchmark of Rogue-Gym is sufficiently fair. In ourexperiments, we evaluate a standard reinforcement learning method, PPO, withand without enhancements for generalization. The results show that someenhancements believed to be effective fail to mitigate the overfitting inRogue-Gym, although others slightly improve the generalization ability.