Text-based person search (TBPS) aims to retrieve the images of the target
person from a large image gallery based on a given natural language
description. Existing methods are dominated by training models with parallel
image-text pairs, which are very costly to collect. In this paper, we make the
first attempt to explore TBPS without parallel image-text data ($\mu$-TBPS), in
which only non-parallel images and texts, or even image-only data, can be
adopted. Towards this end, we propose a two-stage framework,
generation-then-retrieval (GTR), to first generate the corresponding pseudo
text for each image and then perform the retrieval in a supervised manner. In
the generation stage, we propose a fine-grained image captioning strategy to
obtain an enriched description of the person image, which firstly utilizes a
set of instruction prompts to activate the off-the-shelf pretrained
vision-language model to capture and generate fine-grained person attributes,
and then converts the extracted attributes into a textual description via the
finetuned large language model or the hand-crafted template. In the retrieval
stage, considering the noise interference of the generated texts for training
model, we develop a confidence score-based training scheme by enabling more
reliable texts to contribute more during the training. Experimental results on
multiple TBPS benchmarks (i.e., CUHK-PEDES, ICFG-PEDES and RSTPReid) show that
the proposed GTR can achieve a promising performance without relying on
parallel image-text data.

本文提出了一种基于生成 - 检索（GTR）框架的文本搜索人物图像方法（TBPS），其中通过细粒度图像字幕和噪声评分训练来生成人物图像的伪文本，实现了不需要并行图像 - 文本数据的高效搜索。实验表明，所提出的方法在多个 TBPS 基准（即 CUHK-PEDES、ICFG-PEDES 和 RSTPReid）上具有很好的性能。