Abstract
Reinforcement Learning methods are capable of solving complex problems, butresulting policies might perform poorly in environments that are even slightlydifferent. In robotics especially, training and deployment conditions oftenvary and data collection is expensive, making retraining undesirable.Simulation training allows for feasible training times, but on the other handsuffers from a reality-gap when applied in real-world settings. This raises theneed of efficient adaptation of policies acting in new environments. Weconsider this as a problem of transferring knowledge within a family of similarMarkov decision processes. For this purpose we assume that Q-functions are generated by somelow-dimensional latent variable. Given such a Q-function, we can find a masterpolicy that can adapt given different values of this latent variable. Ourmethod learns both the generative mapping and an approximate posterior of thelatent variables, enabling identification of policies for new tasks bysearching only in the latent space, rather than the space of all policies. Thelow-dimensional space, and master policy found by our method enables policiesto quickly adapt to new environments. We demonstrate the method on both apendulum swing-up task in simulation, and for simulation-to-real transfer on apushing task.