Abstract
We present Momentum Contrast (MoCo) for unsupervised visual representationlearning. From a perspective on contrastive learning as dictionary look-up, webuild a dynamic dictionary with a queue and a moving-averaged encoder. Thisenables building a large and consistent dictionary on-the-fly that facilitatescontrastive unsupervised learning. MoCo provides competitive results under thecommon linear protocol on ImageNet classification. More importantly, therepresentations learned by MoCo transfer well to downstream tasks. MoCo canoutperform its supervised pre-training counterpart in 7 detection/segmentationtasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by largemargins. This suggests that the gap between unsupervised and supervisedrepresentation learning has been largely closed in many vision tasks.