Recent studies have demonstrated the effectiveness of using large language
language models (LLMs) in passage ranking. The listwise approaches, such as
RankGPT, have become new state-of-the-art in this task. However, the efficiency
of RankGPT models is limited by the maximum context length and relatively high
latency of LLM inference. To address these issues, in this paper, we propose
PE-Rank, leveraging the single passage embedding as a good context compression
for efficient listwise passage reranking. By treating each passage as a special
token, we can directly input passage embeddings into LLMs, thereby reducing
input length. Additionally, we introduce an inference method that dynamically
constrains the decoding space to these special tokens, accelerating the
decoding process. For adapting the model to reranking, we employ listwise
learning to rank loss for training. Evaluation results on multiple benchmarks
demonstrate that PE-Rank significantly improves efficiency in both prefilling
and decoding, while maintaining competitive ranking effectiveness. {The Code is
available at https://github.com/liuqi6777/pe_rank.}

这篇论文介绍了一种名为 PE-Rank 的方法，它利用单个段落嵌入作为有效的上下文压缩，以提高大型语言模型在段落排名中的效率。通过将每个段落视为特殊标记，将段落嵌入直接输入到语言模型中，从而减少输入长度。此外，还介绍了一种动态约束解码空间的推理方法，加速解码过程。通过在多个基准测试上的评估结果表明，PE-Rank 在预填充和解码方面显著提高了效率，同时保持了有竞争力的排名效果。

利用段落嵌入提高大型语言模型的高效排序

Leveraging Passage Embeddings for Efficient Listwise Reranking with  Large Language Models

Passage reranking is a crucial task in many applications, particularly when
dealing with large-scale documents. Traditional neural architectures are
limited in retrieving the best passage for a question because they usually
match the question to each passage separately, seldom considering contextual
information in other passages that can provide comparison and reference
information. This paper presents a list-context attention mechanism to augment
the passage representation by incorporating the list-context information from
other candidates. The proposed coarse-to-fine (C2F) neural retriever addresses
the out-of-memory limitation of the passage attention mechanism by dividing the
list-context modeling process into two sub-processes, allowing for efficient
encoding of context information from a large number of candidate answers. This
method can be generally used to encode context information from any number of
candidate answers in one pass. Different from most multi-stage information
retrieval architectures, this model integrates the coarse and fine rankers into
the joint optimization process, allowing for feedback between the two layers to
update the model simultaneously. Experiments demonstrate the effectiveness of
the proposed approach.

该研究提出了一种列表上下文关注机制，通过将来自其他候选答案的列表上下文信息并入通道表示来增强通道重排的性能。该模型将粗糙和精细排名器整合到联合优化过程中，实现两层之间的反馈以同时更新模型，实验结果表明了该方法的有效性。