Abstract
We present two techniques to improve landmark localization in images frompartially annotated datasets. Our primary goal is to leverage the commonsituation where precise landmark locations are only provided for a small datasubset, but where class labels for classification or regression tasks relatedto the landmarks are more abundantly available. First, we propose the frameworkof sequential multitasking and explore it here through an architecture forlandmark localization where training with class labels acts as an auxiliarysignal to guide the landmark localization on unlabeled data. A key aspect ofour approach is that errors can be backpropagated through a complete landmarklocalization model. Second, we propose and explore an unsupervised learningtechnique for landmark localization based on having a model predict equivariantlandmarks with respect to transformations applied to the image. We show thatthese techniques, improve landmark prediction considerably and can learneffective detectors even when only a small fraction of the dataset has landmarklabels. We present results on two toy datasets and four real datasets, withhands and faces, and report new state-of-the-art on two datasets in the wild,e.g. with only 5\% of labeled images we outperform previous state-of-the-arttrained on the AFLW dataset.