Abstract
Few-shot learning is challenging for learning algorithms that learn each taskin isolation and from scratch. In contrast, meta-learning learns from manyrelated tasks a meta-learner that can learn a new task more accurately andfaster with fewer examples, where the choice of meta-learners is crucial. Inthis paper, we develop Meta-SGD, an SGD-like, easily trainable meta-learnerthat can initialize and adapt any differentiable learner in just one step, onboth supervised learning and reinforcement learning. Compared to the popularmeta-learner LSTM, Meta-SGD is conceptually simpler, easier to implement, andcan be learned more efficiently. Compared to the latest meta-learner MAML,Meta-SGD has a much higher capacity by learning to learn not just the learnerinitialization, but also the learner update direction and learning rate, all ina single meta-learning process. Meta-SGD shows highly competitive performancefor few-shot learning on regression, classification, and reinforcementlearning.