Delay-Aware Model-Based Reinforcement Learning for Continuous Control

  • 2020-05-11 21:13:37
  • Baiming Chen, Mengdi Xu, Liang Li, Ding Zhao
  • 6

Abstract

Action delays degrade the performance of reinforcement learning in manyreal-world systems. This paper proposes a formal definition of delay-awareMarkov Decision Process and proves it can be transformed into standard MDP withaugmented states using the Markov reward process. We develop a delay-awaremodel-based reinforcement learning framework that can incorporate themulti-step delay into the learned system models without learning effort.Experiments with the Gym and MuJoCo platforms show that the proposeddelay-aware model-based algorithm is more efficient in training andtransferable between systems with various durations of delay compared withoff-policy model-free reinforcement learning methods. Codes available at:https://github.com/baimingc/dambrl.

 

Quick Read (beta)

loading the full paper ...