Abstract
Meta-reinforcement learning algorithms can enable autonomous agents, such asrobots, to quickly acquire new behaviors by leveraging prior experience in aset of related training tasks. However, the onerous data requirements ofmeta-training compounded with the challenge of learning from sensory inputssuch as images have made meta-RL challenging to apply to real robotic systems.Latent state models, which learn compact state representations from a sequenceof observations, can accelerate representation learning from visual inputs. Inthis paper, we leverage the perspective of meta-learning as task inference toshow that latent state models can \emph{also} perform meta-learning given anappropriately defined observation space. Building on this insight, we developmeta-RL with latent dynamics (MELD), an algorithm for meta-RL from images thatperforms inference in a latent state model to quickly acquire new skills givenobservations and rewards. MELD outperforms prior meta-RL methods on severalsimulated image-based robotic control problems, and enables a real WidowXrobotic arm to insert an Ethernet cable into new locations given a sparse taskcompletion signal after only $8$ hours of real world meta-training. To ourknowledge, MELD is the first meta-RL algorithm trained in a real-world roboticcontrol setting from images.