Abstract
Due to the realization that deep reinforcement learning algorithms trained onhigh-dimensional tasks can strongly overfit to their training environments,there have been several studies that investigated the generalizationperformance of these algorithms. However, there has been no similar study thatevaluated the generalization performance of algorithms that were specificallydesigned for generalization, i.e. meta-reinforcement learning algorithms. Inthis paper, we assess the generalization performance of these algorithms byleveraging high-dimensional, procedurally generated environments. We find thatthese algorithms can display strong overfitting when they are evaluated onchallenging tasks. We also observe that scalability to high-dimensional taskswith sparse rewards remains a significant problem among many of the currentmeta-reinforcement learning algorithms. With these results, we highlight theneed for developing meta-reinforcement learning algorithms that can bothgeneralize and scale.