Abstract
In recent years, deep learning-based approaches for visual-inertial odometry(VIO) have shown remarkable performance outperforming traditional geometricmethods. Yet, all existing methods use both the visual and inertialmeasurements for every pose estimation incurring potential computationalredundancy. While visual data processing is much more expensive than that forthe inertial measurement unit (IMU), it may not always contribute to improvingthe pose estimation accuracy. In this paper, we propose an adaptivedeep-learning based VIO method that reduces computational redundancy byopportunistically disabling the visual modality. Specifically, we train apolicy network that learns to deactivate the visual feature extractor on thefly based on the current motion state and IMU readings. A Gumbel-Softmax trickis adopted to train the policy network to make the decision processdifferentiable for end-to-end system training. The learned strategy isinterpretable, and it shows scenario-dependent decision patterns for adaptivecomplexity reduction. Experiment results show that our method achieves asimilar or even better performance than the full-modality baseline with up to78.8% computational complexity reduction for KITTI dataset evaluation. Our codewill be shared in https://github.com/mingyuyng/Visual-Selective-VIO