Visible-infrared person re-identification (V-I ReID) seeks to match images of
individuals captured over a distributed network of RGB and IR cameras. The task
is challenging due to the significant differences between V and I modalities,
especially under real-world conditions, where images are corrupted by, e.g,
blur, noise, and weather. Indeed, state-of-art V-I ReID models cannot leverage
corrupted modality information to sustain a high level of accuracy. In this
paper, we propose an efficient model for multimodal V-I ReID -- named
Multimodal Middle Stream Fusion (MMSF) -- that preserves modality-specific
knowledge for improved robustness to corrupted multimodal images. In addition,
three state-of-art attention-based multimodal fusion models are adapted to
address corrupted multimodal data in V-I ReID, allowing to dynamically balance
each modality importance. Recently, evaluation protocols have been proposed to
assess the robustness of ReID models under challenging real-world scenarios.
However, these protocols are limited to unimodal V settings. For realistic
evaluation of multimodal (and cross-modal) V-I person ReID models, we propose
new challenging corrupted datasets for scenarios where V and I cameras are
co-located (CL) and not co-located (NCL). Finally, the benefits of our Masking
and Local Multimodal Data Augmentation (ML-MDA) strategy are explored to
improve the robustness of ReID models to multimodal corruption. Our experiments
on clean and corrupted versions of the SYSU-MM01, RegDB, and ThermalWORLD
datasets indicate the multimodal V-I ReID models that are more likely to
perform well in real-world operational conditions. In particular, our ML-MDA is
an important strategy for a V-I person ReID system to sustain high accuracy and
robustness when processing corrupted multimodal images. Also, our multimodal
ReID model MMSF outperforms every method under CL and NCL camera scenarios.

本文提出一种名为 具有多种模态中间流融合（MMSF）的高效模型，用于多模态可见光红外人物再识别，旨在提高对损坏的多模态图像的鲁棒性。同时，我们还提出新的可扩展的受损数据集和策略，以提高 V-I 个人再识别系统在现实世界操作条件下的准确性与鲁棒性。