The focus of recent meta-learning research has been on the development oflearning algorithms that can quickly adapt to test time tasks with limited dataand low computational cost. Few-shot learning is widely used as one of thestandard benchmarks in meta-learning. In this work, we show that a simplebaseline: learning a supervised or self-supervised representation on themeta-training set, followed by training a linear classifier on top of thisrepresentation, outperforms state-of-the-art few-shot learning methods. Anadditional boost can be achieved through the use of self-distillation. Thisdemonstrates that using a good learned embedding model can be more effectivethan sophisticated meta-learning algorithms. We believe that our findingsmotivate a rethinking of few-shot image classification benchmarks and theassociated role of meta-learning algorithms. Code is available at:http://github.com/WangYueFt/rfs/.