Abstract
Coordinating the motion between lower and upper limbs and aligning limbcontrol with perception are substantial challenges in robotics, particularly indynamic environments. To this end, we introduce an approach for enabling leggedmobile manipulators to play badminton, a task that requires precisecoordination of perception, locomotion, and arm swinging. We propose a unifiedreinforcement learning-based control policy for whole-body visuomotor skillsinvolving all degrees of freedom to achieve effective shuttlecock tracking andstriking. This policy is informed by a perception noise model that utilizesreal-world camera data, allowing for consistent perception error levels betweensimulation and deployment and encouraging learned active perception behaviors.Our method includes a shuttlecock prediction model, constrained reinforcementlearning for robust motion control, and integrated system identificationtechniques to enhance deployment readiness. Extensive experimental results in avariety of environments validate the robot's capability to predict shuttlecocktrajectories, navigate the service area effectively, and execute precisestrikes against human players, demonstrating the feasibility of using leggedmobile manipulators in complex and dynamic sports scenarios.