Abstract
`In-the-wild' mobile manipulation aims to deploy robots in diverse real-worldenvironments, which requires the robot to (1) have skills that generalizeacross object configurations; (2) be capable of long-horizon task execution indiverse environments; and (3) perform complex manipulation beyondpick-and-place. Quadruped robots with manipulators hold promise for extendingthe workspace and enabling robust locomotion, but existing results do notinvestigate such a capability. This paper proposes WildLMa with threecomponents to address these issues: (1) adaptation of learned low-levelcontroller for VR-enabled whole-body teleoperation and traversability; (2)WildLMa-Skill -- a library of generalizable visuomotor skills acquired viaimitation learning or heuristics and (3) WildLMa-Planner -- an interface oflearned skills that allow LLM planners to coordinate skills for long-horizontasks. We demonstrate the importance of high-quality training data by achievinghigher grasping success rate over existing RL baselines using only tens ofdemonstrations. WildLMa exploits CLIP for language-conditioned imitationlearning that empirically generalizes to objects unseen in trainingdemonstrations. Besides extensive quantitative evaluation, we qualitativelydemonstrate practical robot applications, such as cleaning up trash inuniversity hallways or outdoor terrains, operating articulated objects, andrearranging items on a bookshelf.