Abstract
Vision-language-action (VLA) models have demonstrated strong semanticunderstanding and zero-shot generalization, yet most existing systems assume anaccurate low-level controller with hand-crafted action "vocabulary" such asend-effector pose or root velocity. This assumption confines prior work toquasi-static tasks and precludes the agile, whole-body behaviors required byhumanoid whole-body control (WBC) tasks. To capture this gap in the literature,we start by introducing the first sim-to-real-ready, vision-language,closed-loop benchmark for humanoid WBC, comprising over 150 tasks from 10categories. We then propose LeVERB: Latent Vision-Language-Encoded RobotBehavior, a hierarchical latent instruction-following framework for humanoidvision-language WBC, the first of its kind. At the top level, a vision-languagepolicy learns a latent action vocabulary from synthetically rendered kinematicdemonstrations; at the low level, a reinforcement-learned WBC policy consumesthese latent verbs to generate dynamics-level commands. In our benchmark,LeVERB can zero-shot attain a 80% success rate on simple visual navigationtasks, and 58.5% success rate overall, outperforming naive hierarchicalwhole-body VLA implementation by 7.8 times.