Abstract
General-purpose robotic systems must master a large repertoire of diverseskills to be useful in a range of daily tasks. While reinforcement learningprovides a powerful framework for acquiring individual behaviors, the timeneeded to acquire each skill makes the prospect of a generalist robot trainedwith RL daunting. In this paper, we study how a large-scale collective roboticlearning system can acquire a repertoire of behaviors simultaneously, sharingexploration, experience, and representations across tasks. In this frameworknew tasks can be continuously instantiated from previously learned tasksimproving overall performance and capabilities of the system. To instantiatethis system, we develop a scalable and intuitive framework for specifying newtasks through user-provided examples of desired outcomes, devise a multi-robotcollective learning system for data collection that simultaneously collectsexperience for multiple tasks, and develop a scalable and generalizablemulti-task deep reinforcement learning method, which we call MT-Opt. Wedemonstrate how MT-Opt can learn a wide range of skills, including semanticpicking (i.e., picking an object from a particular category), placing intovarious fixtures (e.g., placing a food item onto a plate), covering, aligning,and rearranging. We train and evaluate our system on a set of 12 real-worldtasks with data collected from 7 robots, and demonstrate the performance of oursystem both in terms of its ability to generalize to structurally similar newtasks, and acquire distinct new tasks more quickly by leveraging pastexperience. We recommend viewing the videos athttps://karolhausman.github.io/mt-opt/