We introduce TextWorld, a sandbox learning environment for the training andevaluation of RL agents on text-based games. TextWorld is a Python library thathandles interactive play-through of text games, as well as backend functionslike state tracking and reward assignment. It comes with a curated list ofgames whose features and challenges we have analyzed. More significantly, itenables users to handcraft or automatically generate new games. Its generativemechanisms give precise control over the difficulty, scope, and language ofconstructed games, and can be used to relax challenges inherent to commercialtext games like partial observability and sparse rewards. By generating sets ofvaried but similar games, TextWorld can also be used to study generalizationand transfer learning. We cast text-based games in the Reinforcement Learningformalism, use our framework to develop a set of benchmark games, and evaluateseveral baseline agents on this set and the curated list.