Recent studies have demonstrated the effectiveness of using large language
language models (LLMs) in passage ranking. The listwise approaches, such as
RankGPT, have become new state-of-the-art in this task. However, the efficiency
of RankGPT models is limited by the maximum context length and relatively high
latency of LLM inference. To address these issues, in this paper, we propose
PE-Rank, leveraging the single passage embedding as a good context compression
for efficient listwise passage reranking. By treating each passage as a special
token, we can directly input passage embeddings into LLMs, thereby reducing
input length. Additionally, we introduce an inference method that dynamically
constrains the decoding space to these special tokens, accelerating the
decoding process. For adapting the model to reranking, we employ listwise
learning to rank loss for training. Evaluation results on multiple benchmarks
demonstrate that PE-Rank significantly improves efficiency in both prefilling
and decoding, while maintaining competitive ranking effectiveness. {The Code is
available at https://github.com/liuqi6777/pe_rank.}

这篇论文介绍了一种名为 PE-Rank 的方法，它利用单个段落嵌入作为有效的上下文压缩，以提高大型语言模型在段落排名中的效率。通过将每个段落视为特殊标记，将段落嵌入直接输入到语言模型中，从而减少输入长度。此外，还介绍了一种动态约束解码空间的推理方法，加速解码过程。通过在多个基准测试上的评估结果表明，PE-Rank 在预填充和解码方面显著提高了效率，同时保持了有竞争力的排名效果。