GOATS: Goal Sampling Adaptation for Scooping with Curriculum Reinforcement Learning

  • 2023-03-09 11:45:48
  • Yaru Niu, Shiyu Jin, Zeqing Zhang, Jiacheng Zhu, Ding Zhao, Liangjun Zhang
  • 1


In this work, we first formulate the problem of goal-conditioned roboticwater scooping with reinforcement learning. This task is challenging due to thecomplex dynamics of fluid and multi-modal goal-reaching. The policy is requiredto achieve both position goals and water amount goals, which leads to a largeconvoluted goal state space. To address these challenges, we introduce GoalSampling Adaptation for Scooping (GOATS), a curriculum reinforcement learningmethod that can learn an effective and generalizable policy for robot scoopingtasks. Specifically, we use a goal-factorized reward formulation andinterpolate position goal distributions and amount goal distributions to createcurriculum through the learning process. As a result, our proposed method canoutperform the baselines in simulation and achieves 5.46% and 8.71% amounterrors on bowl scooping and bucket scooping tasks, respectively, under 1000variations of initial water states in the tank and a large goal state space.Besides being effective in simulation environments, our method can efficientlygeneralize to noisy real-robot water-scooping scenarios with different physicalconfigurations and unseen settings, demonstrating superior efficacy andgeneralizability. The videos of this work are available on our project page:https://sites.google.com/view/goatscooping.


Quick Read (beta)

loading the full paper ...