A key challenge in visible-infrared person re-identification (V-I ReID) is
training a backbone model capable of effectively addressing the significant
discrepancies across modalities. State-of-the-art methods that generate a
single intermediate bridging domain are often less effective, as this generated
domain may not adequately capture sufficient common discriminant information.
This paper introduces the Bidirectional Multi-step Domain Generalization
(BMDG), a novel approach for unifying feature representations across diverse
modalities. BMDG creates multiple virtual intermediate domains by finding and
aligning body part features extracted from both I and V modalities. Indeed,
BMDG aims to reduce the modality gaps in two steps. First, it aligns modalities
in feature space by learning shared and modality-invariant body part prototypes
from V and I images. Then, it generalizes the feature representation by
applying bidirectional multi-step learning, which progressively refines feature
representations in each step and incorporates more prototypes from both
modalities. In particular, our method minimizes the cross-modal gap by
identifying and aligning shared prototypes that capture key discriminative
features across modalities, then uses multiple bridging steps based on this
information to enhance the feature representation. Experiments conducted on
challenging V-I ReID datasets indicate that our BMDG approach outperforms
state-of-the-art part-based models or methods that generate an intermediate
domain from V-I person ReID.

该研究提出了一种名为 Bidirectional Multi-step Domain Generalization (BMDG) 的新方法，用于在不同的感官模态中统一特征表示，通过学习共享和模态不变的人体部位原型来对齐特征空间，然后利用多步学习逐步改进特征表示，并在此基础上使用多个桥接步骤来增强特征表示，有效地减少了感官模态之间的差距。实验证明，BMDG 方法在具有挑战性的可见 - 红外人物重新识别数据集上优于现有的基于部位的模型或通过可见 - 红外人物重新识别生成中间域的方法。

可见 - 红外人员重新识别的双向多步领域泛化

Bidirectional Multi-Step Domain Generalization for Visible-Infrared  Person Re-Identification

In vision-and-language navigation (VLN), an embodied agent is required to
navigate in realistic 3D environments following natural language instructions.
One major bottleneck for existing VLN approaches is the lack of sufficient
training data, resulting in unsatisfactory generalization to unseen
environments. While VLN data is typically collected manually, such an approach
is expensive and prevents scalability. In this work, we address the data
scarcity issue by proposing to automatically create a large-scale VLN dataset
from 900 unlabeled 3D buildings from HM3D. We generate a navigation graph for
each building and transfer object predictions from 2D to generate pseudo 3D
object labels by cross-view consistency. We then fine-tune a pretrained
language model using pseudo object labels as prompts to alleviate the
cross-modal gap in instruction generation. Our resulting HM3D-AutoVLN dataset
is an order of magnitude larger than existing VLN datasets in terms of
navigation environments and instructions. We experimentally demonstrate that
HM3D-AutoVLN significantly increases the generalization ability of resulting
VLN models. On the SPL metric, our approach improves over state of the art by
7.1% and 8.1% on the unseen validation splits of REVERIE and SOON datasets
respectively.

本研究提出通过使用 900 个未标记的 3D 建筑从 HM3D 中自动创建 VLN 数据集，并通过预训练的语言模型对数据集进行微调，从而解决 VLN 方法中的数据稀缺性问题，从而大幅提高 VLN 模型的泛化能力。实验表明该方法在 REVERIE 和 SOON 数据集验证数据集上提高了 7.1% 和 8.1% 的 SPL 性能，取得了良好效果。