Abstract
Transfer learning has proven to be a successful technique to train deeplearning models in the domains where little training data is available. Thedominant approach is to pretrain a model on a large generic dataset such asImageNet and finetune its weights on the target domain. However, in the new eraof an ever-increasing number of massive datasets, selecting the relevant datafor pretraining is a critical issue. We introduce Neural Data Server (NDS), alarge-scale search engine for finding the most useful transfer learning data tothe target domain. Our NDS consists of a dataserver which indexes several largepopular image datasets, and aims to recommend data to a client, an end-userwith a target application with its own small labeled dataset. As in any searchengine that serves information to possibly numerous users, we want the onlinecomputation performed by the dataserver to be minimal. The dataserverrepresents large datasets with a much more compact mixture-of experts model,and employs it to perform data search in a series of dataserver-clienttransactions at a low computational cost. We show the effectiveness of NDS invarious transfer learning scenarios, demonstrating state-of-the-art performanceon several target datasets and tasks such as image classification, objectdetection and instance segmentation. Our Neural Data Server is available as aweb-service at http://aidemos.cs.toronto.edu/nds/, recommending data to userswith the aim to improve performance of their A.I. application.