Text-based person re-identification (TBPReID) aims to retrieve person images
represented by a given textual query. In this task, how to effectively align
images and texts globally and locally is a crucial challenge. Recent works have
obtained high performances by solving Masked Language Modeling (MLM) to align
image/text parts. However, they only performed uni-directional (i.e., from
image to text) local-matching, leaving room for improvement by introducing
opposite-directional (i.e., from text to image) local-matching. In this work,
we introduce Bidirectional Local-Matching (BiLMa) framework that jointly
optimize MLM and Masked Image Modeling (MIM) in TBPReID model training. With
this framework, our model is trained so as the labels of randomly masked both
image and text tokens are predicted by unmasked tokens. In addition, to narrow
the semantic gap between image and text in MIM, we propose Semantic MIM
(SemMIM), in which the labels of masked image tokens are automatically given by
a state-of-the-art human parser. Experimental results demonstrate that our
BiLMa framework with SemMIM achieves state-of-the-art Rank@1 and mAP scores on
three benchmarks.

通过引入双向本地匹配 (BiLMa) 框架，我们在文本输入的图像识别任务中提出了 Masked Language Modeling (MLM) 和 Masked Image Modeling (MIM) 的联合优化方法，同时引入 Semantic MIM (SemMIM) 来减小图像和文本之间的语义差距。实验证明，我们的 BiLMa 框架与 SemMIM 在三个评估基准上达到了最先进的 Rank@1 和 mAP 得分。