Zero-shot sketch-based image retrieval (ZS-SBIR) is challenging due to the
cross-domain nature of sketches and photos, as well as the semantic gap between
seen and unseen image distributions. Previous methods fine-tune pre-trained
models with various side information and learning strategies to learn a compact
feature space that (\romannumeral1) is shared between the sketch and photo
domains and (\romannumeral2) bridges seen and unseen classes. However, these
efforts are inadequate in adapting domains and transferring knowledge from seen
to unseen classes. In this paper, we present an effective \emph{``Adapt and
Align''} approach to address the key challenges. Specifically, we insert simple
and lightweight domain adapters to learn new abstract concepts of the sketch
domain and improve cross-domain representation capabilities. Inspired by recent
advances in image-text foundation models (\textit{e.g.}, CLIP) on zero-shot
scenarios, we explicitly align the learned image embedding with a more semantic
text embedding to achieve the desired knowledge transfer from seen to unseen
classes. Extensive experiments on three benchmark datasets and two popular
backbones demonstrate the superiority of our method in terms of retrieval
accuracy and flexibility.

本文提出了一种有效的 “适应和对齐” 方法来解决无人监督草图图像检索（ZS-SBIR）中的关键挑战，并通过与更语义化的文本嵌入相对齐实现了从已见类到未见类的知识转移。