Abstract
Multispectral images of color-thermal pairs have shown more effective than asingle color channel for pedestrian detection, especially under challengingillumination conditions. However, there is still a lack of studies on how tofuse the two modalities effectively. In this paper, we deeply compare sixdifferent convolutional network fusion architectures and analyse theiradaptations, enabling a vanilla architecture to obtain detection performancescomparable to the state-of-the-art results. Further, we discover thatpedestrian detection confidences from color or thermal images are correlatedwith illumination conditions. With this in mind, we propose anIllumination-aware Faster R-CNN (IAF RCNN). Specifically, an Illumination-awareNetwork is introduced to give an illumination measure of the input image. Thenwe adaptively merge color and thermal sub-networks via a gate function definedover the illumination value. The experimental results on KAIST MultispectralPedestrian Benchmark validate the effectiveness of the proposed IAF R-CNN.