Listwise rerankers based on large language models (LLM) are the zero-shot
state-of-the-art. However, current works in this direction all depend on the
GPT models, making it a single point of failure in scientific reproducibility.
Moreover, it raises the concern that the current research findings only hold
for GPT models but not LLM in general. In this work, we lift this pre-condition
and build for the first time effective listwise rerankers without any form of
dependency on GPT. Our passage retrieval experiments show that our best list se
reranker surpasses the listwise rerankers based on GPT-3.5 by 13% and achieves
97% effectiveness of the ones built on GPT-4. Our results also show that the
existing training datasets, which were expressly constructed for pointwise
ranking, are insufficient for building such listwise rerankers. Instead,
high-quality listwise ranking data is required and crucial, calling for further
work on building human-annotated listwise data resources.

通过消除对 GPT 模型的依赖，本研究首次构建了有效的零阶科学可复现性的无需 GPT 依赖的排序器，能够在通行回收实验中表现出 13% 超越基于 GPT-3.5 的排序器，并达到了基于 GPT-4 构建的排序器的 97% 有效性。研究结果还显示，现有的点对点排序训练数据对于构建此类基于列表的排序器是不足够的，需要高质量的基于列表的排序数据，呼吁进一步建立人工标注的列表数据资源。