Multimodal fusion detection always places high demands on the imaging system
and image pre-processing, while either a high-quality pre-registration system
or image registration processing is costly. Unfortunately, the existing fusion
methods are designed for registered source images, and the fusion of
inhomogeneous features, which denotes a pair of features at the same spatial
location that expresses different semantic information, cannot achieve
satisfactory performance via these methods. As a result, we propose IA-VFDnet,
a CNN-Transformer hybrid learning framework with a unified high-quality
multimodal feature matching module (AKM) and a fusion module (WDAF), in which
AKM and DWDAF work in synergy to perform high-quality infrared-aware visible
fusion detection, which can be applied to smoke and wildfire detection.
Furthermore, experiments on the M3FD dataset validate the superiority of the
proposed method, with IA-VFDnet achieving the best detection performance than
other state-of-the-art methods under conventional registered conditions. In
addition, the first unregistered multimodal smoke and wildfire detection
benchmark is openly available in this letter.

我们提出了一种 CNN-Transformer 混合学习框架（称为 IA-VFDnet），其中包括统一的高质量多模态特征匹配模块（AKM）和融合模块（WDAF），可以应用于烟雾和野火检测，并通过对 M3FD 数据集的实验证明了其在传统注册条件下的最佳检测性能。此外，此论文还公开提供了第一个未注册的多模态烟雾和野火检测基准。