Pre-training has emerged as an effective technique for learning powerful
person representations. Most existing methods have shown that pre-training on
pure-vision large-scale datasets like ImageNet and LUPerson has achieved
remarkable performance. However, solely relying on visual information, the
absence of robust explicit indicators poses a challenge for these methods to
learn discriminative person representations. Drawing inspiration from the
intrinsic fine-grained attribute indicators of person descriptions, we explore
introducing the language modality into person representation learning. To this
end, we propose a novel language-image pre-training framework for person
representation learning, termed PLIP. To explicitly build fine-grained
cross-modal associations, we specifically design three pretext tasks, \ie
semantic-fused image colorization, visual-fused attributes prediction, and
vision-language matching. In addition, due to the lack of an appropriate
dataset, we present a large-scale person dataset named SYNTH-PEDES, where the
Stylish Pedestrian Attributes-union Captioning method is proposed to synthesize
diverse textual descriptions. We pre-train PLIP on SYNTH-PEDES and evaluate our
model by spanning downstream tasks such as text-based Re-ID, image-based Re-ID,
and person attribute recognition. Extensive experiments demonstrate that our
model not only significantly improves existing methods on all these tasks, but
also shows great ability in the few-shot and domain generalization settings.
The code, dataset and weights will be released
at~https://github.com/Zplusdragon/PLIP

该研究提出了一种新的预训练框架用于人物表征学习，名为 PLIP，其中包括三个预文本任务：图像着色、属性预测和视觉 - 语言匹配，在一个新的大规模人物数据集 SYNTH-PEDES 上对其进行了评估并取得了优于先前方法的效果。