Image-text retrieval requires the system to bridge the heterogenous gap
between vision and language for accurate retrieval while keeping the network
lightweight-enough for efficient retrieval. Existing trade-off solutions mainly
study from the view of incorporating cross-modal interactions with the
independent-embedding framework or leveraging stronger pretrained encoders,
which still demand time-consuming similarity measurement or heavyweight model
structure in the retrieval stage. In this work, we propose an image-text
alignment module SelfAlign on top of the independent-embedding framework, which
improves the retrieval accuracy while maintains the retrieval efficiency
without extra supervision. SelfAlign contains two collaborative sub-modules
that force image-text alignment at both concept level and context level by
self-supervised contrastive learning. It does not require cross-modal embedding
interactions during training while maintaining independent image and text
encoders during retrieval. With comparable time cost, SelfAlign consistently
boosts the accuracy of state-of-the-art non-pretraining independent-embedding
models respectively by 9.1%, 4.2% and 6.6% in terms of R@sum score on
Flickr30K, MSCOCO 1K and MS-COCO 5K datasets. The retrieval accuracy also
outperforms most existing interactive-embedding models with orders of magnitude
decrease in retrieval time. The source code is available at:
this https URL

在这项工作中，我们在独立嵌入框架之上提出了一个图像 - 文本对齐模块 SelfAlign，通过自监督对比学习在概念级和语境级强制进行图像 - 文本对齐，提高了检索准确性同时保持了检索效率。