Abstract
Deep reinforcement learning (RL) has emerged as a promising approach forautonomously acquiring complex behaviors from low level sensor observations.Although a large portion of deep RL research has focused on applications invideo games and simulated control, which does not connect with the constraintsof learning in real environments, deep RL has also demonstrated promise inenabling physical robots to learn complex skills in the real world. At the sametime,real world robotics provides an appealing domain for evaluating suchalgorithms, as it connects directly to how humans learn; as an embodied agentin the real world. Learning to perceive and move in the real world presentsnumerous challenges, some of which are easier to address than others, and someof which are often not considered in RL research that focuses only on simulateddomains. In this review article, we present a number of case studies involvingrobotic deep RL. Building off of these case studies, we discuss commonlyperceived challenges in deep RL and how they have been addressed in theseworks. We also provide an overview of other outstanding challenges, many ofwhich are unique to the real-world robotics setting and are not often the focusof mainstream RL research. Our goal is to provide a resource both forroboticists and machine learning researchers who are interested in furtheringthe progress of deep RL in the real world.