Abstract
Both the design and control of a robot play equally important roles in itstask performance. However, while optimal control is well studied in the machinelearning and robotics community, less attention is placed on finding theoptimal robot design. This is mainly because co-optimizing design and controlin robotics is characterized as a challenging problem, and more importantly, acomprehensive evaluation benchmark for co-optimization does not exist. In thispaper, we propose Evolution Gym, the first large-scale benchmark forco-optimizing the design and control of soft robots. In our benchmark, eachrobot is composed of different types of voxels (e.g., soft, rigid, actuators),resulting in a modular and expressive robot design space. Our benchmarkenvironments span a wide range of tasks, including locomotion on various typesof terrains and manipulation. Furthermore, we develop several robotco-evolution algorithms by combining state-of-the-art design optimizationmethods and deep reinforcement learning techniques. Evaluating the algorithmson our benchmark platform, we observe robots exhibiting increasingly complexbehaviors as evolution progresses, with the best evolved designs solving manyof our proposed tasks. Additionally, even though robot designs are evolvedautonomously from scratch without prior knowledge, they often grow to resembleexisting natural creatures while outperforming hand-designed robots.Nevertheless, all tested algorithms fail to find robots that succeed in ourhardest environments. This suggests that more advanced algorithms are requiredto explore the high-dimensional design space and evolve increasinglyintelligent robots -- an area of research in which we hope Evolution Gym willaccelerate progress. Our website with code, environments, documentation, andtutorials is available at http://evogym.csail.mit.edu.