One of the core problems in large-scale recommendations is to retrieve top
relevant candidates accurately and efficiently, preferably in sub-linear time.
Previous approaches are mostly based on a two-step procedure: first learn an
inner-product model, and then use some approximate nearest neighbor (ANN)
search algorithm to find top candidates. In this paper, we present Deep
Retrieval (DR), to learn a retrievable structure directly with user-item
interaction data (e.g. clicks) without resorting to the Euclidean space
assumption in ANN algorithms. DR's structure encodes all candidate items into a
discrete latent space. Those latent codes for the candidates are model
parameters and learnt together with other neural network parameters to maximize
the same objective function. With the model learnt, a beam search over the
structure is performed to retrieve the top candidates for reranking.
Empirically, we first demonstrate that DR, with sub-linear computational
complexity, can achieve almost the same accuracy as the brute-force baseline on
two public datasets. Moreover, we show that, in a live production
recommendation system, a deployed DR approach significantly outperforms a
well-tuned ANN baseline in terms of engagement metrics. To the best of our
knowledge, DR is among the first non-ANN algorithms successfully deployed at
the scale of hundreds of millions of items for industrial recommendation
systems.

本文中介绍了 Deep Retrieval (DR) 算法，通过学习可检索结构来获取最优推荐结果，其编码所有候选项入离散潜在空间，接着使用当前模型下的光束搜索来检索前置候选项，以实现重新排名。实验证明，使用 DR 算法，在两个公共数据集上能够实现近乎于暴力基线的准确性，并且在实时生产推荐系统上，高度优化的 DR 算法显著优于 ANN 基准线。DR 是非 ANN 的推荐算法在工业推荐系统规模的首批成功部署之一。