Abstract
Offline goal-conditioned reinforcement learning (GCRL) is a major problem inreinforcement learning (RL) because it provides a simple, unsupervised, anddomain-agnostic way to acquire diverse behaviors and representations fromunlabeled data without rewards. Despite the importance of this setting, we lacka standard benchmark that can systematically evaluate the capabilities ofoffline GCRL algorithms. In this work, we propose OGBench, a new, high-qualitybenchmark for algorithms research in offline goal-conditioned RL. OGBenchconsists of 8 types of environments, 85 datasets, and reference implementationsof 6 representative offline GCRL algorithms. We have designed these challengingand realistic environments and datasets to directly probe differentcapabilities of algorithms, such as stitching, long-horizon reasoning, and theability to handle high-dimensional inputs and stochasticity. Whilerepresentative algorithms may rank similarly on prior benchmarks, ourexperiments reveal stark strengths and weaknesses in these differentcapabilities, providing a strong foundation for building new algorithms.Project page: https://seohong.me/projects/ogbench