This paper introduces PAG-a novel optimization and decoding approach that
guides autoregressive generation of document identifiers in generative
retrieval models through simultaneous decoding. To this aim, PAG constructs a
set-based and sequential identifier for each document. Motivated by the
bag-of-words assumption in information retrieval, the set-based identifier is
built on lexical tokens. The sequential identifier, on the other hand, is
obtained via quantizing relevance-based representations of documents. Extensive
experiments on MSMARCO and TREC Deep Learning Track data reveal that PAG
outperforms the state-of-the-art generative retrieval model by a large margin
(e.g., 15.6% MRR improvements on MS MARCO), while achieving 22x speed up in
terms of query latency.

本文介绍了 PAG，一种新的优化和解码方法，通过同时解码来指导生成式检索模型中文档标识符的自回归生成。PAG 为每个文档构建了一种基于集合和顺序的标识符，其中基于集合的标识符基于词汇标记建立，而顺序标识符则通过量化文档的基于相关性的表示获得。在 MSMARCO 和 TREC 深度学习赛道数据上进行的大量实验证明 PAG 在性能上超过了现有的生成式检索模型（例如，在 MS MARCO 上提高了 15.6% 的 MRR），同时查询延迟方面达到了 22 倍的加速。

生成式检索中的先行规划：通过同时解码引导自回归生成

Planning Ahead in Generative Retrieval: Guiding Autoregressive  Generation through Simultaneous Decoding

Retrieving keywords (bidwords) with the same intent as query, referred to as
close variant keywords, is of prime importance for effective targeted search
advertising. For head and torso search queries, sponsored search engines use a
huge repository of same intent queries and keywords, mined ahead of time.
Online, this repository is used to rewrite the query and then lookup the
rewrite in a repository of bid keywords contributing to significant revenue.
Recently generative retrieval models have been shown to be effective at the
task of generating such query rewrites. We observe two main limitations of such
generative models. First, rewrites generated by these models exhibit low
lexical diversity, and hence the rewrites fail to retrieve relevant keywords
that have diverse linguistic variations. Second, there is a misalignment
between the training objective - the likelihood of training data, v/s what we
desire - improved quality and coverage of rewrites. In this work, we introduce
CLOVER, a framework to generate both high-quality and diverse rewrites by
optimizing for human assessment of rewrite quality using our diversity-driven
reinforcement learning algorithm. We use an evaluation model, trained to
predict human judgments, as the reward function to finetune the generation
policy. We empirically show the effectiveness of our proposed approach through
offline experiments on search queries across geographies spanning three major
languages. We also perform online A/B experiments on Bing, a large commercial
search engine, which shows (i) better user engagement with an average increase
in clicks by 12.83% accompanied with an average defect reduction by 13.97%, and
(ii) improved revenue by 21.29%.

本文提出了 CLOVER 框架，通过增加多样性的强化学习算法来优化人类对重写质量的评估，从而生成高质量和多样化的查询重写，离线实验和 Bing 上的在线实验都证明了该方法的有效性。