Abstract
Curriculum learning has been a quiet yet crucial component of many of thehigh-profile successes of reinforcement learning. Despite this, none of themajor reinforcement learning libraries directly support curriculum learning orinclude curriculum learning implementations. These methods can improve thecapabilities and robustness of RL agents, but often require significant,complex changes to agent training code. We introduce Syllabus, a library fortraining RL agents with curriculum learning, as a solution to this problem.Syllabus provides a universal API for curriculum learning algorithms,implementations of popular curriculum learning methods, and infrastructure foreasily integrating them with distributed training code written in nearly any RLlibrary. Syllabus provides a minimal API for each of the core components ofcurriculum learning, dramatically simplifying the process of designing newalgorithms and applying existing algorithms to new environments. We demonstratethat the same Syllabus code can be used to train agents written in multipledifferent RL libraries on numerous domains. In doing so, we present the firstexamples of curriculum learning in NetHack and Neural MMO, two of the premierchallenges for single-agent and multi-agent RL respectively, achieving strongresults compared to state of the art baselines.