Action Space Shaping in Deep Reinforcement Learning

  • 2020-04-02 13:25:55
  • Anssi Kanervisto, Christian Scheller, Ville Hautamäki
  • 13

Abstract

Reinforcement learning (RL) has been successful in training agents in variouslearning environments, including video-games. However, such work modifies andshrinks the action space from the game's original. This is to avoid trying"pointless" actions and to ease the implementation. Currently, this is mostlydone based on intuition, with little systematic research supporting the designdecisions. In this work, we aim to gain insight on these action spacemodifications by conducting extensive experiments in video-game environments.Our results show how domain-specific removal of actions and discretization ofcontinuous actions can be crucial for successful learning. With these insights,we hope to ease the use of RL in new environments, by clarifying whataction-spaces are easy to learn.

 

Quick Read (beta)

loading the full paper ...