Abstract
A key question in Reinforcement Learning is which representation an agent canlearn to efficiently reuse knowledge between different tasks. Recently theSuccessor Representation was shown to have empirical benefits for transferringknowledge between tasks with shared transition dynamics. This paper presentsModel Features: a feature representation that clusters behaviourally equivalentstates and that is equivalent to a Model-Reduction. Further, we present aSuccessor Feature model which shows that learning Successor Features isequivalent to learning a Model-Reduction. A novel optimization objective isdeveloped and we provide bounds showing that minimizing this objective resultsin an increasingly improved approximation of a Model-Reduction. Further, weprovide transfer experiments on randomly generated MDPs which vary in theirtransition and reward functions but approximately preserve behaviouralequivalence between states. These results demonstrate that Model Features aresuitable for transfer between tasks with varying transition and rewardfunctions.