Generative Retrieval introduces a new approach to Information Retrieval by
reframing it as a constrained generation task, leveraging recent advancements
in Autoregressive (AR) language models. However, AR-based Generative Retrieval
methods suffer from high inference latency and cost compared to traditional
dense retrieval techniques, limiting their practical applicability. This paper
investigates fully Non-autoregressive (NAR) language models as a more efficient
alternative for generative retrieval. While standard NAR models alleviate
latency and cost concerns, they exhibit a significant drop in retrieval
performance (compared to AR models) due to their inability to capture
dependencies between target tokens. To address this, we question the
conventional choice of limiting the target token space to solely words or
sub-words. We propose PIXAR, a novel approach that expands the target
vocabulary of NAR models to include multi-word entities and common phrases (up
to 5 million tokens), thereby reducing token dependencies. PIXAR employs
inference optimization strategies to maintain low inference latency despite the
significantly larger vocabulary. Our results demonstrate that PIXAR achieves a
relative improvement of 31.0% in MRR@10 on MS MARCO and 23.2% in Hits@5 on
Natural Questions compared to standard NAR models with similar latency and
cost. Furthermore, online A/B experiments on a large commercial search engine
show that PIXAR increases ad clicks by 5.08% and revenue by 4.02%.

该论文研究了非自回归语言模型作为生成式检索的更高效的替代方法，并提出了 PIXAR，一种扩展目标词汇的新方法，通过包括多词实体和常见短语（高达 500 万个标记）来减少标记之间的依赖性，并通过推理优化策略在保持低推理延迟的同时实现了好于标准非自回归模型 31.0% 的相对改进 (MRR@10，MS MARCO 数据集)。