Abstract
Recovering the spatial layout of the cameras and the geometry of the scenefrom extreme-view images is a longstanding challenge in computer vision.Prevailing 3D reconstruction algorithms often adopt the image matching paradigmand presume that a portion of the scene is co-visible across images, yieldingpoor performance when there is little overlap among inputs. In contrast, humanscan associate visible parts in one image to the corresponding invisiblecomponents in another image via prior knowledge of the shapes. Inspired by thisfact, we present a novel concept called virtual correspondences (VCs). VCs area pair of pixels from two images whose camera rays intersect in 3D. Similar toclassic correspondences, VCs conform with epipolar geometry; unlike classiccorrespondences, VCs do not need to be co-visible across views. Therefore VCscan be established and exploited even if images do not overlap. We introduce amethod to find virtual correspondences based on humans in the scene. Weshowcase how VCs can be seamlessly integrated with classic bundle adjustment torecover camera poses across extreme views. Experiments show that our methodsignificantly outperforms state-of-the-art camera pose estimation methods inchallenging scenarios and is comparable in the traditional densely capturedsetup. Our approach also unleashes the potential of multiple downstream taskssuch as scene reconstruction from multi-view stereo and novel view synthesis inextreme-view scenarios.