Abstract
TorchBeast is a platform for reinforcement learning (RL) research in PyTorch.It implements a version of the popular IMPALA algorithm for fast, asynchronous,parallel training of RL agents. Additionally, TorchBeast has simplicity as anexplicit design goal: We provide both a pure-Python implementation("MonoBeast") as well as a multi-machine high-performance version("PolyBeast"). In the latter, parts of the implementation are written in C++,but all parts pertaining to machine learning are kept in simple Python usingPyTorch, with the environments provided using the OpenAI Gym interface. Thisenables researchers to conduct scalable RL research using TorchBeast withoutany programming knowledge beyond Python and PyTorch. In this paper, we describethe TorchBeast design principles and implementation and demonstrate that itperforms on-par with IMPALA on Atari. TorchBeast is released as an open-sourcepackage under the Apache 2.0 license and is available at\url{https://github.com/facebookresearch/torchbeast}.