Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark

  • 2021-03-29 05:00:14
  • Sharada Mohanty, Jyotish Poonganam, Adrien Gaidon, Andrey Kolobov, Blake Wulfe, Dipam Chakraborty, Gražvydas Šemetulskis, João Schapke, Jonas Kubilius, Jurgis Pašukonis, Linas Klimas, Matthew Hausknecht, Patrick MacAlpine, Quang Nhat Tran, Thomas Tumiel, Xiaocheng Tang, Xinwei Chen, Christopher Hesse, Jacob Hilton, William Hebgen Guss, Sahika Genc, John Schulman, Karl Cobbe
  • 0

Abstract

The NeurIPS 2020 Procgen Competition was designed as a centralized benchmarkwith clearly defined tasks for measuring Sample Efficiency and Generalizationin Reinforcement Learning. Generalization remains one of the most fundamentalchallenges in deep reinforcement learning, and yet we do not have enoughbenchmarks to measure the progress of the community on Generalization inReinforcement Learning. We present the design of a centralized benchmark forReinforcement Learning which can help measure Sample Efficiency andGeneralization in Reinforcement Learning by doing end to end evaluation of thetraining and rollout phases of thousands of user submitted code bases in ascalable way. We designed the benchmark on top of the already existing ProcgenBenchmark by defining clear tasks and standardizing the end to end evaluationsetups. The design aims to maximize the flexibility available for researcherswho wish to design future iterations of such benchmarks, and yet imposesnecessary practical constraints to allow for a system like this to scale. Thispaper presents the competition setup and the details and analysis of the topsolutions identified through this setup in context of 2020 iteration of thecompetition at NeurIPS.

 

Quick Read (beta)

loading the full paper ...