Abstract
Reinforcement learning is applied to solve actual complex tasks fromhigh-dimensional, sensory inputs. The last decade has developed a long list ofreinforcement learning algorithms. Recent progress benefits from deep learningfor raw sensory signal representation. One question naturally arises: how welldo they perform concerning different robotic manipulation tasks? Benchmarks useobjective performance metrics to offer a scientific way to compare algorithms.In this paper, we present RMBench, the first benchmark for roboticmanipulations, which have high-dimensional continuous action and state spaces.We implement and evaluate reinforcement learning algorithms that directly useobserved pixels as inputs. We report their average performance and learningcurves to show their performance and stability of training. Our study concludesthat none of the studied algorithms can handle all tasks well, softActor-Critic outperforms most algorithms in average reward and stability, andan algorithm combined with data augmentation may facilitate learning policies.Our code is publicly available athttps://github.com/xiangyanfei212/RMBench-2022, including all benchmark tasksand studied algorithms.