With explosive growth of data volume and ever-increasing diversity of data
modalities, cross-modal similarity search, which conducts nearest neighbor
search across different modalities, has been attracting increasing interest.
This paper presents a deep compact code learning solution for efficient
cross-modal similarity search. Many recent studies have proven that
quantization-based approaches perform generally better than hashing-based
approaches on single-modal similarity search. In this paper, we propose a deep
quantization approach, which is among the early attempts of leveraging deep
neural networks into quantization-based cross-modal similarity search. Our
approach, dubbed shared predictive deep quantization (SPDQ), explicitly
formulates a shared subspace across different modalities and two private
subspaces for individual modalities, and representations in the shared subspace
and the private subspaces are learned simultaneously by embedding them to a
reproducing kernel Hilbert space, where the mean embedding of different
modality distributions can be explicitly compared. In addition, in the shared
subspace, a quantizer is learned to produce the semantics preserving compact
codes with the help of label alignment. Thanks to this novel network
architecture in cooperation with supervised quantization training, SPDQ can
preserve intramodal and intermodal similarities as much as possible and greatly
reduce quantization error. Experiments on two popular benchmarks corroborate
that our approach outperforms state-of-the-art methods.

本文提出了一种共享预测深度量化（SPDQ）模型，通过嵌入到重现核希尔伯特空间中同时学习共享子空间和两个私有子空间的表示，以进行不同模态之间的最近邻搜索，并在标签对齐的帮助下实现语义保留。实验结果显示，该模型在两个流行的基准测试中优于现有的基准方法。

共享的跨模态深度量化预测

Shared Predictive Cross-Modal Deep Quantization

Cross-modal similarity search is a problem about designing a search system
supporting querying across content modalities, e.g., using an image to search
for texts or using a text to search for images. This paper presents a compact
coding solution for efficient search, with a focus on the quantization approach
which has already shown the superior performance over the hashing solutions in
the single-modal similarity search. We propose a cross-modal quantization
approach, which is among the early attempts to introduce quantization into
cross-modal search. The major contribution lies in jointly learning the
quantizers for both modalities through aligning the quantized representations
for each pair of image and text belonging to a document. In addition, our
approach simultaneously learns the common space for both modalities in which
quantization is conducted to enable efficient and effective search using the
Euclidean distance computed in the common space with fast distance table
lookup. Experimental results compared with several competitive algorithms over
three benchmark datasets demonstrate that the proposed approach achieves the
state-of-the-art performance.

本文给出了一种紧凑的编码方案，使用量化方法实现跨模态搜索，通过联合学习图像和文本的量化器来实现。经过实验证明，该方法在三个基准数据集上实现了最先进的性能表现。