Abstract
Deep reinforcement learning has recently seen huge success across multipleareas in the robotics domain. Owing to the limitations of gathering real-worlddata, i.e., sample inefficiency and the cost of collecting it, simulationenvironments are utilized for training the different agents. This not only aidsin providing a potentially infinite data source, but also alleviates safetyconcerns with real robots. Nonetheless, the gap between the simulated and realworlds degrades the performance of the policies once the models are transferredinto real robots. Multiple research efforts are therefore now being directedtowards closing this sim-to-real gap and accomplish more efficient policytransfer. Recent years have seen the emergence of multiple methods applicableto different domains, but there is a lack, to the best of our knowledge, of acomprehensive review summarizing and putting into context the differentmethods. In this survey paper, we cover the fundamental background behindsim-to-real transfer in deep reinforcement learning and overview the mainmethods being utilized at the moment: domain randomization, domain adaptation,imitation learning, meta-learning and knowledge distillation. We categorizesome of the most relevant recent works, and outline the main applicationscenarios. Finally, we discuss the main opportunities and challenges of thedifferent approaches and point to the most promising directions.