Abstract
In this work, we introduce dual goal representations for goal-conditionedreinforcement learning (GCRL). A dual goal representation characterizes a stateby "the set of temporal distances from all other states"; in other words, itencodes a state through its relations to every other state, measured bytemporal distance. This representation provides several appealing theoreticalproperties. First, it depends only on the intrinsic dynamics of the environmentand is invariant to the original state representation. Second, it containsprovably sufficient information to recover an optimal goal-reaching policy,while being able to filter out exogenous noise. Based on this concept, wedevelop a practical goal representation learning method that can be combinedwith any existing GCRL algorithm. Through diverse experiments on the OGBenchtask suite, we empirically show that dual goal representations consistentlyimprove offline goal-reaching performance across 20 state- and pixel-basedtasks.