Abstract

While auxiliary tasks play a key role in shaping the representations learntby reinforcement learning agents, much is still unknown about the mechanismsthrough which this is achieved. This work develops our understanding of therelationship between auxiliary tasks, environment structure, andrepresentations by analysing the dynamics of temporal difference algorithms.Through this approach, we establish a connection between the spectraldecomposition of the transition operator and the representations induced by avariety of auxiliary tasks. We then leverage insights from these theoreticalresults to inform the selection of auxiliary tasks for deep reinforcementlearning agents in sparse-reward environments.

 

Quick Read (beta)