Abstract
While current benchmark reinforcement learning (RL) tasks have been useful todrive progress in the field, they are in many ways poor substitutes forlearning with real-world data. By testing increasingly complex RL algorithms onlow-complexity simulation environments, we often end up with brittle RLpolicies that generalize poorly beyond the very specific domain. To combatthis, we propose three new families of benchmark RL domains that contain someof the complexity of the natural world, while still supporting fast andextensive data acquisition. The proposed domains also permit a characterizationof generalization through fair train/test separation, and easy comparison andreplication of results. Through this work, we challenge the RL researchcommunity to develop more robust algorithms that meet high standards ofevaluation.