Abstract
Learning-based point cloud registration approaches have significantlyoutperformed their traditional counterparts. However, they typically requireextensive training on specific datasets. In this paper, we propose , the firstzero-shot point cloud registration approach that eliminates the need fortraining on point cloud datasets. The cornerstone of ZeroReg is the noveltransfer of image features from keypoints to the point cloud, enriched byaggregating information from 3D geometric neighborhoods. Specifically, weextract keypoints and features from 2D image pairs using a frozen pretrained 2Dbackbone. These features are then projected in 3D, and patches are constructedby searching for neighboring points. We integrate the geometric and visualfeatures of each point using our novel parameter-free geometric decoder.Subsequently, the task of determining correspondences between point clouds isformulated as an optimal transport problem. Extensive evaluations of ZeroRegdemonstrate its competitive performance against both traditional andlearning-based methods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet,ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%,respectively.