Abstract
Reinforcement Learning (RL) has achieved tremendous success in many complexdecision-making tasks. However, safety concerns are raised during deploying RLin real-world applications, leading to a growing demand for safe RL algorithms,such as in autonomous driving and robotics scenarios. While safe control has along history, the study of safe RL algorithms is still in the early stages. Toestablish a good foundation for future safe RL research, in this paper, weprovide a review of safe RL from the perspectives of methods, theories, andapplications. Firstly, we review the progress of safe RL from five dimensionsand come up with five crucial problems for safe RL being deployed in real-worldapplications, coined as "2H3W". Secondly, we analyze the algorithm and theoryprogress from the perspectives of answering the "2H3W" problems. Particularly,the sample complexity of safe RL algorithms is reviewed and discussed, followedby an introduction to the applications and benchmarks of safe RL algorithms.Finally, we open the discussion of the challenging problems in safe RL, hopingto inspire future research on this thread. To advance the study of safe RLalgorithms, we release an open-sourced repository containing theimplementations of major safe RL algorithms at the link:https://github.com/chauncygu/Safe-Reinforcement-Learning-Baselines.git.