Sketch-based image retrieval (SBIR) is widely recognized as an important
vision problem which implies a wide range of real-world applications. Recently,
research interests arise in solving this problem under the more realistic and
challenging setting of zero-shot learning. In this paper, we investigate this
problem from the viewpoint of domain adaptation which we show is critical in
improving feature embedding in the zero-shot scenario. Based on a framework
which starts with a pre-trained model on ImageNet and fine-tunes it on the
training set of SBIR benchmark, we advocate the importance of preserving
previously acquired knowledge, e.g., the rich discriminative features learned
from ImageNet, to improve the model's transfer ability. For this purpose, we
design an approach named Semantic-Aware Knowledge prEservation (SAKE), which
fine-tunes the pre-trained model in an economical way and leverages semantic
information, e.g., inter-class relationship, to achieve the goal of knowledge
preservation. Zero-shot experiments on two extended SBIR datasets, TU-Berlin
and Sketchy, verify the superior performance of our approach. Extensive
diagnostic experiments validate that knowledge preserved benefits SBIR in
zero-shot settings, as a large fraction of the performance gain is from the
more properly structured feature embedding for photo images. Code is available
at: this https URL

本文提出了一种名为 Semantic-Aware Knowledge prEservation (SAKE) 的方法，在预训练模型的基础上通过 fine-tunes 和利用语义信息来保留图像检索中先前获得的知识，从而在零样本学习的场景下实现更好的性能表现。