Abstract
In the context of noisy partial label learning (NPLL), each training sampleis associated with a set of candidate labels annotated by multiple noisyannotators. With the emergence of high-performance pre-trained vision-languagemodels (VLMs) such as CLIP, LLaVA and GPT-4V, the direction of using thesemodels to replace time-consuming manual annotation workflows and achieve``manual-annotation-free" training for downstream tasks has become a highlypromising research avenue. This paper focuses on learning from noisy partiallabels annotated by pre-trained VLMs and proposes an innovative collaborativeconsistency regularization (Co-Reg) method. Unlike the symmetric noiseprimarily addressed in traditional noisy label learning, the noise generated bypre-trained models is instance-dependent, embodying the underlying patterns ofthe pre-trained models themselves, which significantly increases the learningdifficulty for the model. To address this, we simultaneously train two neuralnetworks that implement collaborative purification of training labels through a``Co-Pseudo-Labeling" mechanism, while enforcing consistency regularizationconstraints in both the label space and feature representation space.Specifically, we construct multiple anti-overfitting mechanisms thatefficiently mine latent information from noisy partially labeled samplesincluding alternating optimization of contrastive feature representations andpseudo-labels, as well as maintaining prototypical class vectors in the sharedfeature space.