Text-to-image person re-identification (ReID) aims to retrieve images of a
person based on a given textual description. The key challenge is to learn the
relations between detailed information from visual and textual modalities.
Existing works focus on learning a latent space to narrow the modality gap and
further build local correspondences between two modalities. However, these
methods assume that image-to-text and text-to-image associations are
modality-agnostic, resulting in suboptimal associations. In this work, we show
the discrepancy between image-to-text association and text-to-image association
and propose CADA: Cross-Modal Adaptive Dual Association that finely builds
bidirectional image-text detailed associations. Our approach features a
decoder-based adaptive dual association module that enables full interaction
between visual and textual modalities, allowing for bidirectional and adaptive
cross-modal correspondence associations. Specifically, the paper proposes a
bidirectional association mechanism: Association of text Tokens to image
Patches (ATP) and Association of image Regions to text Attributes (ARA). We
adaptively model the ATP based on the fact that aggregating cross-modal
features based on mistaken associations will lead to feature distortion. For
modeling the ARA, since the attributes are typically the first distinguishing
cues of a person, we propose to explore the attribute-level association by
predicting the masked text phrase using the related image region. Finally, we
learn the dual associations between texts and images, and the experimental
results demonstrate the superiority of our dual formulation. Codes will be made
publicly available.

本文提出了 CADA：交叉模态自适应双向关联机制，通过文本到图像和图像到文本的双向关联来实现文本到图像的人物再识别，实验证明了该双向关联机制的优越性。