Abstract
Despite recent successes of reinforcement learning (RL), it remains achallenge for agents to transfer learned skills to related environments. Tofacilitate research addressing this problem, we propose CausalWorld, abenchmark for causal structure and transfer learning in a robotic manipulationenvironment. The environment is a simulation of an open-source roboticplatform, hence offering the possibility of sim-to-real transfer. Tasks consistof constructing 3D shapes from a given set of blocks - inspired by how childrenlearn to build complex structures. The key strength of CausalWorld is that itprovides a combinatorial family of such tasks with common causal structure andunderlying factors (including, e.g., robot and object masses, colors, sizes).The user (or the agent) may intervene on all causal variables, which allows forfine-grained control over how similar different tasks (or task distributions)are. One can thus easily define training and evaluation distributions of adesired difficulty level, targeting a specific form of generalization (e.g.,only changes in appearance or object mass). Further, this commonparametrization facilitates defining curricula by interpolating between aninitial and a target task. While users may define their own task distributions,we present eight meaningful distributions as concrete benchmarks, ranging fromsimple to very challenging, all of which require long-horizon planning as wellas precise low-level motor control. Finally, we provide baseline results for asubset of these tasks on distinct training curricula and correspondingevaluation protocols, verifying the feasibility of the tasks in this benchmark.