Text-based Person Search (TBPS) aims to retrieve images of target pedestrian
indicated by textual descriptions. It is essential for TBPS to extract
fine-grained local features and align them crossing modality. Existing methods
utilize external tools or heavy cross-modal interaction to achieve explicit
alignment of cross-modal fine-grained features, which is inefficient and
time-consuming. In this work, we propose a Vision-Guided Semantic-Group Network
(VGSG) for text-based person search to extract well-aligned fine-grained visual
and textual features. In the proposed VGSG, we develop a Semantic-Group Textual
Learning (SGTL) module and a Vision-guided Knowledge Transfer (VGKT) module to
extract textual local features under the guidance of visual local clues. In
SGTL, in order to obtain the local textual representation, we group textual
features from the channel dimension based on the semantic cues of language
expression, which encourages similar semantic patterns to be grouped implicitly
without external tools. In VGKT, a vision-guided attention is employed to
extract visual-related textual features, which are inherently aligned with
visual cues and termed vision-guided textual features. Furthermore, we design a
relational knowledge transfer, including a vision-language similarity transfer
and a class probability transfer, to adaptively propagate information of the
vision-guided textual features to semantic-group textual features. With the
help of relational knowledge transfer, VGKT is capable of aligning
semantic-group textual features with corresponding visual features without
external tools and complex pairwise interaction. Experimental results on two
challenging benchmarks demonstrate its superiority over state-of-the-art
methods.

通过提出一种视觉引导的语义组网络 (VGSG)，本文针对基于文本的人物搜索 (TBPS) 问题，在引导的视觉线索下提取了对齐良好的细粒度视觉和文本特征，并设计了一种关系性知识传递方法来适应地传播信息从而在无需外部工具和复杂配对交互的情况下将语义组文本特征与相应的视觉特征进行对齐，该方法在两个具有挑战性的基准测试中展现了优越性。

VGSG: 基于视觉引导的文本人物检索的语义组网络

VGSG: Vision-Guided Semantic-Group Network for Text-based Person Search

Existing person re-identification (ReID) methods typically directly load the
pre-trained ImageNet weights for initialization. However, as a fine-grained
classification task, ReID is more challenging and exists a large domain gap
between ImageNet classification. Inspired by the great success of
self-supervised representation learning with contrastive objectives, in this
paper, we design an Unsupervised Pre-training framework for ReID based on the
contrastive learning (CL) pipeline, dubbed UP-ReID. During the pre-training, we
attempt to address two critical issues for learning fine-grained ReID features:
(1) the augmentations in CL pipeline may distort the discriminative clues in
person images. (2) the fine-grained local features of person images are not
fully-explored. Therefore, we introduce an intra-identity
(I$^2$-)regularization in the UP-ReID, which is instantiated as two constraints
coming from global image aspect and local patch aspect: a global consistency is
enforced between augmented and original person images to increase robustness to
augmentation, while an intrinsic contrastive constraint among local patches of
each image is employed to fully explore the local discriminative clues.
Extensive experiments on multiple popular Re-ID datasets, including PersonX,
Market1501, CUHK03, and MSMT17, demonstrate that our UP-ReID pre-trained model
can significantly benefit the downstream ReID fine-tuning and achieve
state-of-the-art performance. Codes and models will be released to
this https URL

本文设计了一个基于对比学习管道的无监督预训练框架 UP-ReID，旨在解决细粒度 ReID 特征学习中的两个关键问题，并介绍了全局一致性和局部补丁之间的内在对比约束，在多个流行的 Re-ID 数据集上进行了广泛的实验，结果表明，我们的 UP-ReID 预训练模型可以显著改进下游 ReID 微调，并实现了最先进的性能。