Abstract
We present PlanarRecon -- a novel framework for globally coherent detectionand reconstruction of 3D planes from a posed monocular video. Unlike previousworks that detect planes in 2D from a single image, PlanarRecon incrementallydetects planes in 3D for each video fragment, which consists of a set of keyframes, from a volumetric representation of the scene using neural networks. Alearning-based tracking and fusion module is designed to merge planes fromprevious fragments to form a coherent global plane reconstruction. Such designallows PlanarRecon to integrate observations from multiple views within eachfragment and temporal information across different ones, resulting in anaccurate and coherent reconstruction of the scene abstraction withlow-polygonal geometry. Experiments show that the proposed approach achievesstate-of-the-art performances on the ScanNet dataset while being real-time.