Multispectral images of color-thermal pairs have shown more effective than a
single color channel for pedestrian detection, especially under challenging
illumination conditions. However, there is still a lack of studies on how to
fuse the two modalities effectively. In this paper, we deeply compare six
different convolutional network fusion architectures and analyse their
adaptations, enabling a vanilla architecture to obtain detection performances
comparable to the state-of-the-art results. Further, we discover that
pedestrian detection confidences from color or thermal images are correlated
with illumination conditions. With this in mind, we propose an
Illumination-aware Faster R-CNN (IAF RCNN). Specifically, an Illumination-aware
Network is introduced to give an illumination measure of the input image. Then
we adaptively merge color and thermal sub-networks via a gate function defined
over the illumination value. The experimental results on KAIST Multispectral
Pedestrian Benchmark validate the effectiveness of the proposed IAF R-CNN.

该研究深入比较了六种不同的卷积网络融合架构，并分析了它们的适应性，从而使传统架构能够实现与最先进的结果可比的检测性能。然后，提出了一种感知光照的 Faster R-CNN，该方法采用适应性门控函数来整合彩色和热图像，实现了对行人的有效检测，并在 KAIST 多光谱行人基准上获得了验证。