Depth-aware Glass Surface Detection with Cross-modal Context Mining

  • 2022-06-22 18:56:09
  • Jiaying Lin, Yuen Hei Yeung, Rynson W. H. Lau
  • 0


Glass surfaces are becoming increasingly ubiquitous as modern buildings tendto use a lot of glass panels. This however poses substantial challenges on theoperations of autonomous systems such as robots, self-driving cars and drones,as the glass panels can become transparent obstacles to the navigation.Existingworks attempt to exploit various cues, including glass boundary context orreflections, as a prior. However, they are all based on input RGB images.Weobserve that the transmission of 3D depth sensor light through glass surfacesoften produces blank regions in the depth maps, which can offer additionalinsights to complement the RGB image features for glass surface detection. Inthis paper, we propose a novel framework for glass surface detection byincorporating RGB-D information, with two novel modules: (1) a cross-modalcontext mining (CCM) module to adaptively learn individual and mutual contextfeatures from RGB and depth information, and (2) a depth-missing awareattention (DAA) module to explicitly exploit spatial locations where missingdepths occur to help detect the presence of glass surfaces. In addition, wepropose a large-scale RGB-D glass surface detection dataset, called\textit{RGB-D GSD}, for RGB-D glass surface detection. Our dataset comprises3,009 real-world RGB-D glass surface images with precise annotations. Extensiveexperimental results show that our proposed model outperforms state-of-the-artmethods.


Quick Read (beta)

loading the full paper ...