Abstract
Adapting deep networks to new concepts from few examples is extremelychallenging, due to the high computational and data requirements of standardfine-tuning procedures. Most works on meta-learning and few-shot learning havethus focused on simple learning techniques for adaptation, such as nearestneighbors or gradient descent. Nonetheless, the machine learning literaturecontains a wealth of methods that learn non-deep models very efficiently. Inthis work we propose to use these fast convergent methods as the mainadaptation mechanism for few-shot learning. The main idea is to teach a deepnetwork to use standard machine learning tools, such as logistic regression, aspart of its own internal model, enabling it to quickly adapt to novel tasks.This requires back-propagating errors through the solver steps. While normallythe matrix operations involved would be costly, the small number of examplesworks to our advantage, by making use of the Woodbury identity. We propose bothiterative and closed-form solvers, based on logistic regression and ridgeregression components. Our methods achieve excellent performance on threefew-shot learning benchmarks, showing competitive performance on Omniglot andsurpassing all state-of-the-art alternatives on miniImageNet and CIFAR-100.