Abstract
Multiview feature learning aims to learn discriminative features byintegrating the distinct information in each view. However, most existingmethods still face significant challenges in learning view-consistencyfeatures, which are crucial for effective multiview learning. Motivated by thetheories of CCA and contrastive learning in multiview feature learning, wepropose the hierarchical consensus network (HCN) in this paper. The HCN derivesthree consensus indices for capturing the hierarchical consensus across views,which are classifying consensus, coding consensus, and global consensus,respectively. Specifically, classifying consensus reinforces class-levelcorrespondence between views from a CCA perspective, while coding consensusclosely resembles contrastive learning and reflects contrastive comparison ofindividual instances. Global consensus aims to extract consensus informationfrom two perspectives simultaneously. By enforcing the hierarchical consensus,the information within each view is better integrated to obtain morecomprehensive and discriminative features. The extensive experimental resultsobtained on four multiview datasets demonstrate that the proposed methodsignificantly outperforms several state-of-the-art methods.