Abstract
Visual Reinforcement Learning is a popular and powerful framework that takesfull advantage of the Deep Learning breakthrough. However, it is also knownthat variations in the input (e.g., different colors of the panorama due to theseason of the year) or the task (e.g., changing the speed limit for a car torespect) could require complete retraining of the agents. In this work, weleverage recent developments in unifying latent representations to demonstratethat it is possible to combine the components of an agent, rather than retrainit from scratch. We build upon the recent relative representations frameworkand adapt it for Visual RL. This allows us to create completely new agentscapable of handling environment-task combinations never seen during training.Our work paves the road toward a more accessible and flexible use ofreinforcement learning.