Abstract
In this work, we first formulate the problem of goal-conditioned roboticwater scooping with reinforcement learning. This task is challenging due to thecomplex dynamics of fluid and multi-modal goal-reaching. The policy is requiredto achieve both position goals and water amount goals, which leads to a largeconvoluted goal state space. To address these challenges, we introduce GoalSampling Adaptation for Scooping (GOATS), a curriculum reinforcement learningmethod that can learn an effective and generalizable policy for robot scoopingtasks. Specifically, we use a goal-factorized reward formulation andinterpolate position goal distributions and amount goal distributions to createcurriculum through the learning process. As a result, our proposed method canoutperform the baselines in simulation and achieves 5.46% and 8.71% amounterrors on bowl scooping and bucket scooping tasks, respectively, under 1000variations of initial water states in the tank and a large goal state space.Besides being effective in simulation environments, our method can efficientlygeneralize to noisy real-robot water-scooping scenarios with different physicalconfigurations and unseen settings, demonstrating superior efficacy andgeneralizability. The videos of this work are available on our project page:https://sites.google.com/view/goatscooping.