Data Distillation: Towards Omni-Supervised Learning

  • 2017-12-12 18:55:57
  • Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, Kaiming He
  • 64

Abstract

We investigate omni-supervised learning, a special regime of semi-supervisedlearning in which the learner exploits all available labeled data plusinternet-scale sources of unlabeled data. Omni-supervised learning islower-bounded by performance on existing labeled datasets, offering thepotential to surpass state-of-the-art fully supervised methods. To exploit theomni-supervised setting, we propose data distillation, a method that ensemblespredictions from multiple transformations of unlabeled data, using a singlemodel, to automatically generate new training annotations. We argue that visualrecognition models have recently become accurate enough that it is now possibleto apply classic ideas about self-training to challenging real-world data. Ourexperimental results show that in the cases of human keypoint detection andgeneral object detection, state-of-the-art models trained with datadistillation surpass the performance of using labeled data from the COCOdataset alone.

 

Quick Read (beta)

loading the full paper ...