Zero-shot learning offers an efficient solution for a machine learning model
to treat unseen categories, avoiding exhaustive data collection. Zero-shot
Sketch-based Image Retrieval (ZS-SBIR) simulates real-world scenarios where it
is hard and costly to collect paired sketch-photo samples. We propose a novel
framework that indirectly aligns sketches and photos by contrasting them
through texts, removing the necessity of access to sketch-photo pairs. With an
explicit modality encoding learned from data, our approach disentangles
modality-agnostic semantics from modality-specific information, bridging the
modality gap and enabling effective cross-modal content retrieval within a
joint latent space. From comprehensive experiments, we verify the efficacy of
the proposed model on ZS-SBIR, and it can be also applied to generalized and
fine-grained settings.

零样本学习在机器学习模型中提供了一种有效的解决方案，用于处理未见类别，避免了繁琐数据收集。本文提出了一种新颖的框架，通过对比文本间接对齐素描和照片，避免了对素描照片成对样本的需求。通过从数据中学习明确的形态编码，我们的方法将形态不可知的语义与形态特定信息进行了分解，桥接了形态间的差距，并在联合潜空间内实现了有效的跨模态内容检索。通过全面的实验验证了所提出模型在零样本基于素描的图像检索上的有效性，并可应用于广义和细粒度设置。