We present Polish Information Retrieval Benchmark (PIRB), a comprehensive
evaluation framework encompassing 41 text information retrieval tasks for
Polish. The benchmark incorporates existing datasets as well as 10 new,
previously unpublished datasets covering diverse topics such as medicine, law,
business, physics, and linguistics. We conduct an extensive evaluation of over
20 dense and sparse retrieval models, including the baseline models trained by
us as well as other available Polish and multilingual methods. Finally, we
introduce a three-step process for training highly effective language-specific
retrievers, consisting of knowledge distillation, supervised fine-tuning, and
building sparse-dense hybrid retrievers using a lightweight rescoring model. In
order to validate our approach, we train new text encoders for Polish and
compare their results with previously evaluated methods. Our dense models
outperform the best solutions available to date, and the use of hybrid methods
further improves their performance.

我们提出了一个综合的评估框架 - 波兰信息检索基准 (PIRB)，其中包括 41 个用于波兰语的文本信息检索任务。该基准涵盖了现有数据集和 10 个涉及医学、法律、商业、物理学和语言学等多样主题的未公开数据集。我们对 20 多个密集和稀疏检索模型进行了广泛评估，包括我们训练的基准模型以及其他现有的波兰语和多语言方法。最后，我们介绍了一个由知识蒸馏、有监督微调和使用轻量级重新评分模型构建的稀疏 - 密集混合检索器的三步训练过程，以提高语言特定的检索器的效果。为了验证我们的方法，我们为波兰语训练了新的文本编码器，并将其结果与先前评估的方法进行了比较。我们的密集模型优于迄今为止最好的解决方案，而混合方法的使用进一步提高了它们的性能。