Abstract
Due to the empirical success of reinforcement learning, an increasing numberof students study the subject. However, from our practical teaching experience,we see students entering the field (bachelor, master and early PhD) oftenstruggle. On the one hand, textbooks and (online) lectures provide thefundamentals, but students find it hard to translate between equations andcode. On the other hand, public codebases do provide practical examples, butthe implemented algorithms tend to be complex, and the underlying testenvironments contain multiple reinforcement learning challenges at once.Although this is realistic from a research perspective, it often hinderseducational conceptual understanding. To solve this issue we introduce EduGym,a set of educational reinforcement learning environments and associatedinteractive notebooks tailored for education. Each EduGym environment isspecifically designed to illustrate a certain aspect/challenge of reinforcementlearning (e.g., exploration, partial observability, stochasticity, etc.), whilethe associated interactive notebook explains the challenge and its possiblesolution approaches, connecting equations and code in a single document. Anevaluation among RL students and researchers shows 86% of them think EduGym isa useful tool for reinforcement learning education. All notebooks are availablefrom https://sites.google.com/view/edu-gym/home, while the full softwarepackage can be installed from https://github.com/RLG-Leiden/edugym.