Abstract
The objective of transfer reinforcement learning is to generalize from a setof previous tasks to unseen new tasks. In this work, we focus on the transferscenario where the dynamics among tasks are the same, but their goals differ.Although general value function (Sutton et al., 2011) has been shown to beuseful for knowledge transfer, learning a universal value function can bechallenging in practice. To attack this, we propose (1) to use universalsuccessor representations (USR) to represent the transferable knowledge and (2)a USR approximator (USRA) that can be trained by interacting with theenvironment. Our experiments show that USR can be effectively applied to newtasks, and the agent initialized by the trained USRA can achieve the goalconsiderably faster than random initialization.