Abstract
In this work, we first formulate the problem of robotic water scooping usinggoal-conditioned reinforcement learning. This task is particularly challengingdue to the complex dynamics of fluid and the need to achieve multi-modal goals.The policy is required to successfully reach both position goals and wateramount goals, which leads to a large convoluted goal state space. To overcomethese challenges, we introduce Goal Sampling Adaptation for Scooping (GOATS), acurriculum reinforcement learning method that can learn an effective andgeneralizable policy for robot scooping tasks. Specifically, we use agoal-factorized reward formulation and interpolate position goal distributionsand amount goal distributions to create curriculum throughout the learningprocess. As a result, our proposed method can outperform the baselines insimulation and achieves 5.46% and 8.71% amount errors on bowl scooping andbucket scooping tasks, respectively, under 1000 variations of initial waterstates in the tank and a large goal state space. Besides being effective insimulation environments, our method can efficiently adapt to noisy real-robotwater-scooping scenarios with diverse physical configurations and unseensettings, demonstrating superior efficacy and generalizability. The videos ofthis work are available on our project page:https://sites.google.com/view/goatscooping.