Abstract
Learning robot tasks or controllers using deep reinforcement learning hasbeen proven effective in simulations. Learning in simulation has severaladvantages. For example, one can fully control the simulated environment,including halting motions while performing computations. Another advantage whenrobots are involved, is that the amount of time a robot is occupied learning atask---rather than being productive---can be reduced by transferring thelearned task to the real robot. Transfer learning requires some amount offine-tuning on the real robot. For tasks which involve complex (non-linear)dynamics, the fine-tuning itself may take a substantial amount of time. Inorder to reduce the amount of fine-tuning we propose to learn robustifiedcontrollers in simulation. Robustified controllers are learned by exploitingthe ability to change simulation parameters (both appearance and dynamics) forsuccessive training episodes. An additional benefit for this approach is thatit alleviates the precise determination of physics parameters for thesimulator, which is a non-trivial task. We demonstrate our proposed approach ona real setup in which a robot aims to solve a maze game, which involves complexdynamics due to static friction and potentially large accelerations. We showthat the amount of fine-tuning in transfer learning for a robustifiedcontroller is substantially reduced compared to a non-robustified controller.