Abstract
The aim of multi-task reinforcement learning is two-fold: (1) efficientlylearn by training against multiple tasks and (2) quickly adapt, using limitedsamples, to a variety of new tasks. In this work, the tasks correspond toreward functions for environments with the same (or similar) dynamical models.We propose to learn a dynamical model during the training process and use thismodel to perform sample-efficient adaptation to new tasks at test time. We usesignificantly fewer samples by performing policy optimization only in a"virtual" environment whose transitions are given by our learned dynamicalmodel. Our algorithm sequentially trains against several tasks. Uponencountering a new task, we first warm-up a policy on our learned dynamicalmodel, which requires no new samples from the environment. We then adapt thedynamical model with samples from this policy in the real environment. Weevaluate our approach on several continuous control benchmarks and demonstrateits efficacy over MAML, a state-of-the-art meta-learning algorithm, on thesetasks.