Abstract
Meta reinforcement learning (meta-RL) is able to accelerate the acquisitionof new tasks by learning from past experience. Current meta-RL methods usuallylearn to adapt to new tasks by directly optimizing the parameters of policiesover primitive actions. However, for complex tasks which requires sophisticatedcontrol strategies, it would be quite inefficient to to directly learn such ameta-policy. Moreover, this problem can become more severe and even fail inspare reward settings, which is quite common in practice. To this end, wepropose a new meta-RL algorithm called meta goal-generation for hierarchical RL(MGHRL) by leveraging hierarchical actor-critic framework. Instead of directlygenerate policies over primitive actions for new tasks, MGHRL learns togenerate high-level meta strategies over subgoals given past experience andleaves the rest of how to achieve subgoals as independent RL subtasks. Ourempirical results on several challenging simulated robotics environments showthat our method enables more efficient and effective meta-learning from pastexperience and outperforms state-of-the-art meta-RL and Hierarchical-RL methodsin sparse reward settings.