This work presents a new strategy for multi-class classification thatrequires no class-specific labels, but instead leverages pairwise similaritybetween examples, which is a weaker form of annotation. The proposed method,meta classification learning, optimizes a binary classifier for pairwisesimilarity prediction and through this process learns a multi-class classifieras a submodule. We formulate this approach, present a probabilistic graphicalmodel for it, and derive a surprisingly simple loss function that can be usedto learn neural network-based models. We then demonstrate that this sameframework generalizes to the supervised, unsupervised cross-task, andsemi-supervised settings. Our method is evaluated against state of the art inall three learning paradigms and shows a superior or comparable accuracy,providing evidence that learning multi-class classification without multi-classlabels is a viable learning option.