Abstract
Traversing risky terrains with sparse footholds poses a significant challengefor humanoid robots, requiring precise foot placements and stable locomotion.Existing approaches designed for quadrupedal robots often fail to generalize tohumanoid robots due to differences in foot geometry and unstable morphology,while learning-based approaches for humanoid locomotion still face greatchallenges on complex terrains due to sparse foothold reward signals andinefficient learning processes. To address these challenges, we introduceBeamDojo, a reinforcement learning (RL) framework designed for enabling agilehumanoid locomotion on sparse footholds. BeamDojo begins by introducing asampling-based foothold reward tailored for polygonal feet, along with a doublecritic to balancing the learning process between dense locomotion rewards andsparse foothold rewards. To encourage sufficient trail-and-error exploration,BeamDojo incorporates a two-stage RL approach: the first stage relaxes theterrain dynamics by training the humanoid on flat terrain while providing itwith task terrain perceptive observations, and the second stage fine-tunes thepolicy on the actual task terrain. Moreover, we implement a onboard LiDAR-basedelevation map to enable real-world deployment. Extensive simulation andreal-world experiments demonstrate that BeamDojo achieves efficient learning insimulation and enables agile locomotion with precise foot placement on sparsefootholds in the real world, maintaining a high success rate even undersignificant external disturbances.