Abstract
Multispectral pedestrian detection has attracted increasing attention fromthe research community due to its crucial competence for many around-the-clockapplications (e.g., video surveillance and autonomous driving), especiallyunder insufficient illumination conditions. We create a human baseline over theKAIST dataset and reveal that there is still a large gap between current topdetectors and human performance. To narrow this gap, we propose a networkfusion architecture, which consists of a multispectral proposal network togenerate pedestrian proposals, and a subsequent multispectral classificationnetwork to distinguish pedestrian instances from hard negatives. The unifiednetwork is learned by jointly optimizing pedestrian detection and semanticsegmentation tasks. The final detections are obtained by integrating theoutputs from different modalities as well as the two stages. The approachsignificantly outperforms state-of-the-art methods on the KAIST dataset whileremain fast. Additionally, we contribute a sanitized version of trainingannotations for the KAIST dataset, and examine the effects caused by differentkinds of annotation errors. Future research of this problem will benefit fromthe sanitized version which eliminates the interference of annotation errors.