Abstract
We study how robots can autonomously learn skills that require a combinationof navigation and grasping. While reinforcement learning in principle providesfor automated robotic skill learning, in practice reinforcement learning in thereal world is challenging and often requires extensive instrumentation andsupervision. Our aim is to devise a robotic reinforcement learning system forlearning navigation and manipulation together, in an autonomous way withouthuman intervention, enabling continual learning under realistic assumptions.Our proposed system, ReLMM, can learn continuously on a real-world platformwithout any environment instrumentation, without human intervention, andwithout access to privileged information, such as maps, objects positions, or aglobal view of the environment. Our method employs a modularized policy withcomponents for manipulation and navigation, where manipulation policyuncertainty drives exploration for the navigation controller, and themanipulation module provides rewards for navigation. We evaluate our method ona room cleanup task, where the robot must navigate to and pick up itemsscattered on the floor. After a grasp curriculum training phase, ReLMM canlearn navigation and grasping together fully automatically, in around 40 hoursof autonomous real-world training.