Unsupervised visible-infrared person re-identification (USL-VI-ReID) is a
promising yet challenging retrieval task. The key challenges in USL-VI-ReID are
to effectively generate pseudo-labels and establish pseudo-label
correspondences across modalities without relying on any prior annotations.
Recently, clustered pseudo-label methods have gained more attention in
USL-VI-ReID. However, previous methods fell short of fully exploiting the
individual nuances, as they simply utilized a single memory that represented an
identity to establish cross-modality correspondences, resulting in ambiguous
cross-modality correspondences. To address the problem, we propose a
Multi-Memory Matching (MMM) framework for USL-VI-ReID. We first design a
Cross-Modality Clustering (CMC) module to generate the pseudo-labels through
clustering together both two modality samples. To associate cross-modality
clustered pseudo-labels, we design a Multi-Memory Learning and Matching (MMLM)
module, ensuring that optimization explicitly focuses on the nuances of
individual perspectives and establishes reliable cross-modality
correspondences. Finally, we design a Soft Cluster-level Alignment (SCA) module
to narrow the modality gap while mitigating the effect of noise pseudo-labels
through a soft many-to-many alignment strategy. Extensive experiments on the
public SYSU-MM01 and RegDB datasets demonstrate the reliability of the
established cross-modality correspondences and the effectiveness of our MMM.
The source codes will be released.

提出了一种用于可见 - 红外无监督人物再识别的多内存匹配框架，通过生成伪标签和建立不依赖于任何先前注释的多模态伪标签对应关系来解决该任务中的关键挑战。通过使用交叉模态聚类、多内存学习和匹配以及软聚类级对齐策略来改进现有方法的不足，提高了跨模态对应关系的可靠性和效果。

无监督可见光 - 红外人物再识别的多记忆匹配

Multi-Memory Matching for Unsupervised Visible-Infrared Person  Re-Identification

The success of supervised learning requires large-scale ground truth labels
which are very expensive, time-consuming, or may need special skills to
annotate. To address this issue, many self- or un-supervised methods are
developed. Unlike most existing self-supervised methods to learn only 2D image
features or only 3D point cloud features, this paper presents a novel and
effective self-supervised learning approach to jointly learn both 2D image
features and 3D point cloud features by exploiting cross-modality and
cross-view correspondences without using any human annotated labels.
Specifically, 2D image features of rendered images from different views are
extracted by a 2D convolutional neural network, and 3D point cloud features are
extracted by a graph convolution neural network. Two types of features are fed
into a two-layer fully connected neural network to estimate the cross-modality
correspondence. The three networks are jointly trained (i.e. cross-modality) by
verifying whether two sampled data of different modalities belong to the same
object, meanwhile, the 2D convolutional neural network is additionally
optimized through minimizing intra-object distance while maximizing
inter-object distance of rendered images in different views (i.e. cross-view).
The effectiveness of the learned 2D and 3D features is evaluated by
transferring them on five different tasks including multi-view 2D shape
recognition, 3D shape recognition, multi-view 2D shape retrieval, 3D shape
retrieval, and 3D part-segmentation. Extensive evaluations on all the five
different tasks across different datasets demonstrate strong generalization and
effectiveness of the learned 2D and 3D features by the proposed self-supervised
method.

通过利用跨模式和视角的对应关系，该论文提出了一种新颖且有效的自监督学习方法，联合学习 2D 图像特征和 3D 点云特征，并通过神经网络评估跨模态的对应关系，而无需使用人工标注标签。