Abstract
In this paper, we consider a highly general image recognition settingwherein, given a labelled and unlabelled set of images, the task is tocategorize all images in the unlabelled set. Here, the unlabelled images maycome from labelled classes or from novel ones. Existing recognition methods arenot able to deal with this setting, because they make several restrictiveassumptions, such as the unlabelled instances only coming from known - orunknown - classes and the number of unknown classes being known a-priori. Weaddress the more unconstrained setting, naming it 'Generalized CategoryDiscovery', and challenge all these assumptions. We first establish strongbaselines by taking state-of-the-art algorithms from novel category discoveryand adapting them for this task. Next, we propose the use of visiontransformers with contrastive representation learning for this open worldsetting. We then introduce a simple yet effective semi-supervised $k$-meansmethod to cluster the unlabelled data into seen and unseen classesautomatically, substantially outperforming the baselines. Finally, we alsopropose a new approach to estimate the number of classes in the unlabelleddata. We thoroughly evaluate our approach on public datasets for generic objectclassification including CIFAR10, CIFAR100 and ImageNet-100, and forfine-grained visual recognition including CUB, Stanford Cars and Herbarium19,benchmarking on this new setting to foster future research.