We propose a Composite Code Sparse Autoencoder (CCSA) approach for
Approximate Nearest Neighbor (ANN) search of document representations based on
Siamese-BERT models. In Information Retrieval (IR), the ranking pipeline is
generally decomposed in two stages: the first stage focus on retrieving a
candidate set from the whole collection. The second stage re-ranks the
candidate set by relying on more complex models. Recently, Siamese-BERT models
have been used as first stage ranker to replace or complement the traditional
bag-of-word models. However, indexing and searching a large document collection
require efficient similarity search on dense vectors and this is why ANN
techniques come into play. Since composite codes are naturally sparse, we first
show how CCSA can learn efficient parallel inverted index thanks to an
uniformity regularizer. Second, CCSA can be used as a binary quantization
method and we propose to combine it with the recent graph based ANN techniques.
Our experiments on MSMARCO dataset reveal that CCSA outperforms IVF with
product quantization. Furthermore, CCSA binary quantization is beneficial for
the index size, and memory usage for the graph-based HNSW method, while
maintaining a good level of recall and MRR. Third, we compare with recent
supervised quantization methods for image retrieval and find that CCSA is able
to outperform them.

本文提出了一种用于文档表示的复合码稀疏自动编码器（CCSA）方法，以近似最近邻（ANN）搜索为基础。该模型可以替代传统的词袋模型，并可与最近的基于图的 ANN 技术相结合。经实验证明，CCSA 在给定数据集上优于 IVF with product quantization，同时，CCSA 的二进制量化对于基于图的 HNSW 方法的索引大小和内存使用有好处，并保持良好的召回率和 MRR。