Query rewriting refers to an established family of approaches that are
applied to underspecified and ambiguous queries to overcome the vocabulary
mismatch problem in document ranking. Queries are typically rewritten during
query processing time for better query modelling for the downstream ranker.
With the advent of large-language models (LLMs), there have been initial
investigations into using generative approaches to generate pseudo documents to
tackle this inherent vocabulary gap. In this work, we analyze the utility of
LLMs for improved query rewriting for text ranking tasks. We find that there
are two inherent limitations of using LLMs as query re-writers -- concept drift
when using only queries as prompts and large inference costs during query
processing. We adopt a simple, yet surprisingly effective, approach called
context aware query rewriting (CAR) to leverage the benefits of LLMs for query
understanding. Firstly, we rewrite ambiguous training queries by context-aware
prompting of LLMs, where we use only relevant documents as context.Unlike
existing approaches, we use LLM-based query rewriting only during the training
phase. Eventually, a ranker is fine-tuned on the rewritten queries instead of
the original queries during training. In our extensive experiments, we find
that fine-tuning a ranker using re-written queries offers a significant
improvement of up to 33% on the passage ranking task and up to 28% on the
document ranking task when compared to the baseline performance of using
original queries.

采用上下文感知查询重写的方法，通过使用大语言模型进行查询理解并对排序器进行微调，能够显著提高查询重写对排名任务的性能。

基于 LLM 的上下文感知查询重写文本排名器

Context Aware Query Rewriting for Text Rankers using LLM

Multi-stage ranking pipelines have been a practical solution in modern search
systems, where the first-stage retrieval is to return a subset of candidate
documents, and latter stages attempt to re-rank those candidates. Unlike
re-ranking stages going through quick technique shifts during past decades, the
first-stage retrieval has long been dominated by classical term-based models.
Unfortunately, these models suffer from the vocabulary mismatch problem, which
may block re-ranking stages from relevant documents at the very beginning.
Therefore, it has been a long-term desire to build semantic models for the
first-stage retrieval that can achieve high recall efficiently. Recently, we
have witnessed an explosive growth of research interests on the first-stage
semantic retrieval models. We believe it is the right time to survey current
status, learn from existing methods, and gain some insights for future
development. In this paper, we describe the current landscape of the
first-stage retrieval models under a unified framework to clarify the
connection between classical term-based retrieval methods, early semantic
retrieval methods and neural semantic retrieval methods. Moreover, we identify
some open challenges and envision some future directions, with the hope of
inspiring more researches on these important yet less investigated topics.

该论文介绍了现代搜索系统中多阶段排名管道的普遍解决方案，阐述了第一阶段检索模型的传统词项模型和语义模型之间的联系，并针对词汇不匹配问题提出了基于语义的解决思路和神经语义检索技术所带来的机遇和挑战。