Abstract
Visual Reinforcement Learning (Visual RL), coupled with high-dimensionalobservations, has consistently confronted the long-standing challenge ofout-of-distribution generalization. Despite the focus on algorithms aimed atresolving visual generalization problems, we argue that the devil is in theexisting benchmarks as they are restricted to isolated tasks and generalizationcategories, undermining a comprehensive evaluation of agents' visualgeneralization capabilities. To bridge this gap, we introduce RL-ViGen: a novelReinforcement Learning Benchmark for Visual Generalization, which containsdiverse tasks and a wide spectrum of generalization types, thereby facilitatingthe derivation of more reliable conclusions. Furthermore, RL-ViGen incorporatesthe latest generalization visual RL algorithms into a unified framework, underwhich the experiment results indicate that no single existing algorithm hasprevailed universally across tasks. Our aspiration is that RL-ViGen will serveas a catalyst in this area, and lay a foundation for the future creation ofuniversal visual generalization RL agents suitable for real-world scenarios.Access to our code and implemented algorithms is provided athttps://gemcollector.github.io/RL-ViGen/.