The text retrieval task is mainly performed in two ways: the bi-encoder
approach and the generative approach. The bi-encoder approach maps the document
and query embeddings to common vector space and performs a nearest neighbor
search. It stably shows high performance and efficiency across different
domains but has an embedding space bottleneck as it interacts in L2 or inner
product space. The generative retrieval model retrieves by generating a target
sequence and overcomes the embedding space bottleneck by interacting in the
parametric space. However, it fails to retrieve the information it has not seen
during the training process as it depends solely on the information encoded in
its own model parameters. To leverage the advantages of both approaches, we
propose Contextualized Generative Retrieval model, which uses contextualized
embeddings (output embeddings of a language model encoder) as vocab embeddings
at the decoding step of generative retrieval. The model uses information
encoded in both the non-parametric space of contextualized token embeddings and
the parametric space of the generative retrieval model. Our approach of
generative retrieval with contextualized vocab embeddings shows higher
performance than generative retrieval with only vanilla vocab embeddings in the
document retrieval task, an average of 6% higher performance in KILT (NQ, TQA)
and 2X higher in NQ-320k, suggesting the benefits of using contextualized
embedding in generative retrieval models.

提出了一种上下文生成检索模型，其在生成式检索模型的解码步骤中使用上下文嵌入，结合了双编码器方法和生成式检索模型的优点，显示出比只有普通嵌入的情况更高的性能。