Abstract
The success of Reinforcement Learning (RL) heavily relies on the ability tolearn robust representations from the observations of the environment. In mostcases, the representations learned purely by the reinforcement learning losscan differ vastly across states depending on how the value functions change.However, the representations learned need not be very specific to the task athand. Relying only on the RL objective may yield representations that varygreatly across successive time steps. In addition, since the RL loss has achanging target, the representations learned would depend on how good thecurrent values/policies are. Thus, disentangling the representations from themain task would allow them to focus more on capturing transition dynamics whichcan improve generalization. To this end, we propose locally constrainedrepresentations, where an auxiliary loss forces the state representations to bepredictable by the representations of the neighbouring states. This encouragesthe representations to be driven not only by the value/policy learning but alsoself-supervised learning, which constrains the representations from changingtoo rapidly. We evaluate the proposed method on several known benchmarks andobserve strong performance. Especially in continuous control tasks, ourexperiments show a significant advantage over a strong baseline.