We study the zero-shot Composed Image Retrieval (ZS-CIR) task, which is to
retrieve the target image given a reference image and a description without
training on the triplet datasets. Previous works generate pseudo-word tokens by
projecting the reference image features to the text embedding space. However,
they focus on the global visual representation, ignoring the representation of
detailed attributes, e.g., color, object number and layout. To address this
challenge, we propose a Knowledge-Enhanced Dual-stream zero-shot composed image
retrieval framework (KEDs). KEDs implicitly models the attributes of the
reference images by incorporating a database. The database enriches the
pseudo-word tokens by providing relevant images and captions, emphasizing
shared attribute information in various aspects. In this way, KEDs recognizes
the reference image from diverse perspectives. Moreover, KEDs adopts an extra
stream that aligns pseudo-word tokens with textual concepts, leveraging
pseudo-triplets mined from image-text pairs. The pseudo-word tokens generated
in this stream are explicitly aligned with fine-grained semantics in the text
embedding space. Extensive experiments on widely used benchmarks, i.e.
ImageNet-R, COCO object, Fashion-IQ and CIRR, show that KEDs outperforms
previous zero-shot composed image retrieval methods.

我们研究了零样本组合图像检索任务，提出了一种知识增强的双流零样本组合图像检索框架（KEDs），该框架通过数据库丰富了伪词令牌，并采用额外的流来对伪词令牌与文本概念进行对齐，从而在细粒度语义中显式地识别出参考图像。大量实验证明，KEDs 优于之前的零样本组合图像检索方法。