Abstract
Simulated virtual environments serve as one of the main driving forces behinddeveloping and evaluating skill learning algorithms. However, existingenvironments typically only simulate rigid body physics. Additionally, thesimulation process usually does not provide gradients that might be useful forplanning and control optimizations. We introduce a new differentiable physicsbenchmark called PasticineLab, which includes a diverse collection of soft bodymanipulation tasks. In each task, the agent uses manipulators to deform theplasticine into the desired configuration. The underlying physics enginesupports differentiable elastic and plastic deformation using the DiffTaichisystem, posing many under-explored challenges to robotic agents. We evaluateseveral existing reinforcement learning (RL) methods and gradient-based methodson this benchmark. Experimental results suggest that 1) RL-based approachesstruggle to solve most of the tasks efficiently; 2) gradient-based approaches,by optimizing open-loop control sequences with the built-in differentiablephysics engine, can rapidly find a solution within tens of iterations, butstill fall short on multi-stage tasks that require long-term planning. Weexpect that PlasticineLab will encourage the development of novel algorithmsthat combine differentiable physics and RL for more complex physics-based skilllearning tasks.