Abstract
Parkour is a grand challenge for legged locomotion that requires robots toovercome various obstacles rapidly in complex environments. Existing methodscan generate either diverse but blind locomotion skills or vision-based butspecialized skills by using reference animal data or complex rewards. However,autonomous parkour requires robots to learn generalizable skills that are bothvision-based and diverse to perceive and react to various scenarios. In thiswork, we propose a system for learning a single end-to-end vision-based parkourpolicy of diverse parkour skills using a simple reward without any referencemotion data. We develop a reinforcement learning method inspired by directcollocation to generate parkour skills, including climbing over high obstacles,leaping over large gaps, crawling beneath low barriers, squeezing through thinslits, and running. We distill these skills into a single vision-based parkourpolicy and transfer it to a quadrupedal robot using its egocentric depthcamera. We demonstrate that our system can empower two different low-costrobots to autonomously select and execute appropriate parkour skills totraverse challenging real-world environments.