Abstract
Many Reinforcement Learning (RL) approaches use joint control signals(positions, velocities, torques) as action space for continuous control tasks.We propose to lift the action space to a higher level in the form of subgoalsfor a motion generator (a combination of motion planner and trajectoryexecutor). We argue that, by lifting the action space and by leveragingsampling-based motion planners, we can efficiently use RL to solve complex,long-horizon tasks that could not be solved with existing RL methods in theoriginal action space. We propose ReLMoGen -- a framework that combines alearned policy to predict subgoals and a motion generator to plan and executethe motion needed to reach these subgoals. To validate our method, we applyReLMoGen to two types of tasks: 1) Interactive Navigation tasks, navigationproblems where interactions with the environment are required to reach thedestination, and 2) Mobile Manipulation tasks, manipulation tasks that requiremoving the robot base. These problems are challenging because they are usuallylong-horizon, hard to explore during training, and comprise alternating phasesof navigation and interaction. Our method is benchmarked on a diverse set ofseven robotics tasks in photo-realistic simulation environments. In allsettings, ReLMoGen outperforms state-of-the-art Reinforcement Learning andHierarchical Reinforcement Learning baselines. ReLMoGen also shows outstandingtransferability between different motion generators at test time, indicating agreat potential to transfer to real robots.