Abstract
We present an approach to estimating camera rotation in crowded, real-worldscenes from handheld monocular video. While camera rotation estimation is awell-studied problem, no previous methods exhibit both high accuracy andacceptable speed in this setting. Because the setting is not addressed well byother datasets, we provide a new dataset and benchmark, with high-accuracy,rigorously verified ground truth, on 17 video sequences. Methods developed forwide baseline stereo (e.g., 5-point methods) perform poorly on monocular video.On the other hand, methods used in autonomous driving (e.g., SLAM) leveragespecific sensor setups, specific motion models, or local optimizationstrategies (lagging batch processing) and do not generalize well to handheldvideo. Finally, for dynamic scenes, commonly used robustification techniqueslike RANSAC require large numbers of iterations, and become prohibitively slow.We introduce a novel generalization of the Hough transform on SO(3) toefficiently and robustly find the camera rotation most compatible with opticalflow. Among comparably fast methods, ours reduces error by almost 50\% over thenext best, and is more accurate than any method, irrespective of speed. Thisrepresents a strong new performance point for crowded scenes, an importantsetting for computer vision. The code and the dataset are available athttps://fabiendelattre.com/robust-rotation-estimation.