Abstract
Image matching, which aims to identify corresponding pixel locations betweenimages, is crucial in a wide range of scientific disciplines, aiding in imageregistration, fusion, and analysis. In recent years, deep learning-based imagematching algorithms have dramatically outperformed humans in rapidly andaccurately finding large amounts of correspondences. However, when dealing withimages captured under different imaging modalities that result in significantappearance changes, the performance of these algorithms often deteriorates dueto the scarcity of annotated cross-modal training data. This limitation hindersapplications in various fields that rely on multiple image modalities to obtaincomplementary information. To address this challenge, we propose a large-scalepre-training framework that utilizes synthetic cross-modal training signals,incorporating diverse data from various sources, to train models to recognizeand match fundamental structures across images. This capability is transferableto real-world, unseen cross-modality image matching tasks. Our key finding isthat the matching model trained with our framework achieves remarkablegeneralizability across more than eight unseen cross-modality registrationtasks using the same network weight, substantially outperforming existingmethods, whether designed for generalization or tailored for specific tasks.This advancement significantly enhances the applicability of image matchingtechnologies across various scientific disciplines and paves the way for newapplications in multi-modality human and artificial intelligence analysis andbeyond.