Abstract
Reinforcement learning (RL) is playing an increasingly important role infields such as robotic control and autonomous driving. However, the gap betweensimulation and the real environment remains a major obstacle to the practicaldeployment of RL. Agents trained in simulators often struggle to maintainperformance when transferred to real-world physical environments. In thispaper, we propose a latent space based approach to analyze the impact ofsimulation on real-world policy improvement in model-based settings. As anatural extension of model-based methods, our approach enables an intuitiveobservation of the challenges faced by model-based methods in sim-to-realtransfer. Experiments conducted in the MuJoCo environment evaluate theperformance of our method in both measuring and mitigating the sim-to-real gap.The experiments also highlight the various challenges that remain in overcomingthe sim-to-real gap, especially for model-based methods.