Abstract
The aim of multi-task reinforcement learning is two-fold: (1) efficientlylearn by training against multiple tasks and (2) quickly adapt, using limitedsamples, to a variety of new tasks. In this work, the tasks correspond toreward functions for environments with the same (or similar) dynamical models.We propose to learn a dynamical model during the training process and use thismodel to perform sample-efficient adaptation to new tasks at test time. Ouralgorithm sequentially trains against several tasks. We use significantly fewersamples by performing policy optimization only in a "virtual" environment whosetransitions are given by our learned dynamical model. Upon encountering a newtask, we first warm-up a policy on our learned dynamical model, which requiresno new samples from the environment. We then adapt the dynamical model withsamples from this policy in the real environment. We evaluate our approach onseveral continuous control benchmarks and demonstrate its efficacy over MAML, astate-of-the-art meta-learning algorithm, on these tasks.