Learning to Design Games: Strategic Environments in Reinforcement Learning

  • 2018-05-23 08:56:12
  • Haifeng Zhang, Jun Wang, Zhiming Zhou, Weinan Zhang, Ying Wen, Yong Yu, Wenxin Li
  • 0

Abstract

In typical reinforcement learning (RL), the environment is assumed given andthe goal of the learning is to identify an optimal policy for the agent takingactions through its interactions with the environment. In this paper, we extendthis setting by considering the environment is not given, but controllable andlearnable through its interaction with the agent at the same time. Thisextension is motivated by environment design scenarios in the real-world,including game design, shopping space design and traffic signal design.Theoretically, we find a dual Markov decision process (MDP) w.r.t. theenvironment to that w.r.t. the agent, and derive a policy gradient solution tooptimizing the parametrized environment. Furthermore, discontinuousenvironments are addressed by a proposed general generative framework. Ourexperiments on a Maze game design task show the effectiveness of the proposedalgorithms in generating diverse and challenging Mazes against various agentsettings.

 

Quick Read (beta)

loading the full paper ...