Efficient Reinforcement Learning with a Mind-Game for Full-Length StarCraft II

  • 2019-03-02 15:02:03
  • Ruo-Ze Liu, Haifeng Guo, Xiaozhong Ji, Yang Yu, Zitai Xiao, Yuzhou Wu, Zhen-Jia Pang, Tong Lu
  • 8


StarCraft II provides an extremely challenging platform for reinforcementlearning due to its huge state-space and game length. The previous fastestmethod requires days to train a full-length game policy in a single commercialmachine. In this paper, we introduce the mind-game to facilitate thereinforcement learning, which is an abstract task model. With the mind-game,the policy is firstly trained in the mind-game fastly and is then mapped to thereal game for the second phase training. In our experiments, the trained agentcan achieve a 100% win-rate on the map Simple64 against the most difficultnon-cheating built-in bot (level-7), and the training is 100 times faster thanthe previous ones under the same computational resource. To test thegeneralization performance of the agent, a Golden level of StarCraft II Ladderhuman player has competed with the agent. With restricted strategy, the agentwins the human player by 4 out of 5 games. The mind-game approach might shedsome light for further studies of efficient reinforcement learning. The codesare publicly available (https://github.com/mindgameSC2/mind-SC2).


Introduction (beta)



Conclusion (beta)