Abstract
Multi-label zero-shot classification aims to predict multiple unseen classlabels for an input image. It is more challenging than its single-labelcounterpart. On one hand, the unconstrained number of labels assigned to eachimage makes the model more easily overfit to those seen classes. On the otherhand, there is a large semantic gap between seen and unseen classes in theexisting multi-label classification datasets. To address these difficultissues, this paper introduces a novel multi-label zero-shot classificationframework by learning to transfer from external knowledge. We observe thatImageNet is commonly used to pretrain the feature extractor and has a large andfine-grained label space. This motivates us to exploit it as external knowledgeto bridge the seen and unseen classes and promote generalization. Specifically,we construct a knowledge graph including not only classes from the targetdataset but also those from ImageNet. Since ImageNet labels are not availablein the target dataset, we propose a novel PosVAE module to infer their initialstates in the extended knowledge graph. Then we design a relational graphconvolutional network (RGCN) to propagate information among classes and achieveknowledge transfer. Experimental results on two benchmark datasets demonstratethe effectiveness of the proposed approach.