Abstract
Reinforcement learning (RL) is a popular paradigm for addressing sequentialdecision tasks in which the agent has only limited environmental feedback.Despite many advances over the past three decades, learning in many domainsstill requires a large amount of interaction with the environment, which can beprohibitively expensive in realistic scenarios. To address this problem,transfer learning has been applied to reinforcement learning such thatexperience gained in one task can be leveraged when starting to learn the next,harder task. More recently, several lines of research have explored how tasks,or data samples themselves, can be sequenced into a curriculum for the purposeof learning a problem that may otherwise be too difficult to learn fromscratch. In this article, we present a framework for curriculum learning (CL)in reinforcement learning, and use it to survey and classify existing CLmethods in terms of their assumptions, capabilities, and goals. Finally, we useour framework to find open problems and suggest directions for future RLcurriculum learning research.