Visual retrieval aims to search for the most relevant visual items, e.g.,
images and videos, from a candidate gallery with a given query item. Accuracy
and efficiency are two competing objectives in retrieval tasks. Instead of
crafting a new method pursuing further improvement on accuracy, in this paper
we propose a multi-teacher distillation framework Whiten-MTD, which is able to
transfer knowledge from off-the-shelf pre-trained retrieval models to a
lightweight student model for efficient visual retrieval. Furthermore, we
discover that the similarities obtained by different retrieval models are
diversified and incommensurable, which makes it challenging to jointly distill
knowledge from multiple models. Therefore, we propose to whiten the output of
teacher models before fusion, which enables effective multi-teacher
distillation for retrieval models. Whiten-MTD is conceptually simple and
practically effective. Extensive experiments on two landmark image retrieval
datasets and one video retrieval dataset demonstrate the effectiveness of our
proposed method, and its good balance of retrieval performance and efficiency.
Our source code is released at this https URL

通过多教师蒸馏框架 Whiten-MTD，该论文提出了将知识从预训练的检索模型转移到轻量级学生模型以实现高效的可视化检索，实验证明了该方法在检索性能和效率上的良好平衡。

全员漂白：高效视觉检索的多教师蒸馏

Let All be Whitened: Multi-teacher Distillation for Efficient Visual  Retrieval

Visual retrieval tasks such as image retrieval and person re-identification
(Re-ID) aim at effectively and thoroughly searching images with similar content
or the same identity. After obtaining retrieved examples, re-ranking is a
widely adopted post-processing step to reorder and improve the initial
retrieval results by making use of the contextual information from semantically
neighboring samples. Prevailing re-ranking approaches update distance metrics
and mostly rely on inefficient crosscheck set comparison operations while
computing expanded neighbors based distances. In this work, we present an
efficient re-ranking method which refines initial retrieval results by updating
features. Specifically, we reformulate re-ranking based on Graph Convolution
Networks (GCN) and propose a novel Graph Convolution based Re-ranking (GCR) for
visual retrieval tasks via feature propagation. To accelerate computation for
large-scale retrieval, a decentralized and synchronous feature propagation
algorithm which supports parallel or distributed computing is introduced. In
particular, the plain GCR is extended for cross-camera retrieval and an
improved feature propagation formulation is presented to leverage affinity
relationships across different cameras. It is also extended for video-based
retrieval, and Graph Convolution based Re-ranking for Video (GCRV) is proposed
by mathematically deriving a novel profile vector generation method for the
tracklet. Without bells and whistles, the proposed approaches achieve
state-of-the-art performances on seven benchmark datasets from three different
tasks, i.e., image retrieval, person Re-ID and video-based person Re-ID.

本文介绍了一种采用图卷积网络的高效接近搜索再排序方法，通过更新特征值来完善视觉检索任务的初步结果，并以状态 - of-the-art 表现获得了在 3 个不同任务的七个基准数据集上的结果。