Abstract
In recent years, reinforcement learning (RL) methods have been applied tomodel gameplay with great success, achieving super-human performance in variousenvironments, such as Atari, Go, and Poker. However, those studies mostly focuson winning the game and have largely ignored the rich and complex humanmotivations, which are essential for understanding different players' diversebehaviors. In this paper, we present a novel method called Multi-MotivationBehavior Modeling (MMBM) that takes the multifaceted human motivations intoconsideration and models the underlying value structure of the players usinginverse RL. Our approach does not require the access to the dynamic of thesystem, making it feasible to model complex interactive environments such asmassively multiplayer online games. MMBM is tested on the World of WarcraftAvatar History dataset, which recorded over 70,000 users' gameplay spanningthree years period. Our model reveals the significant difference of valuestructures among different player groups. Using the results of motivationmodeling, we also predict and explain their diverse gameplay behaviors andprovide a quantitative assessment of how the redesign of the game environmentimpacts players' behaviors.