The goal of the TREC Fair Ranking track was to develop a benchmark for
evaluating retrieval systems in terms of fairness to different content
providers in addition to classic notions of relevance. As part of the
benchmark, we defined standardized fairness metrics with evaluation protocols
and released a dataset for the fair ranking problem. The 2019 task focused on
reranking academic paper abstracts given a query. The objective was to fairly
represent relevant authors from several groups that were unknown at the system
submission time. Thus, the track emphasized the development of systems which
have robust performance across a variety of group definitions. Participants
were provided with querylog data (queries, documents, and relevance) from
Semantic Scholar. This paper presents an overview of the track, including the
task definition, descriptions of the data and the annotation process, as well
as a comparison of the performance of submitted systems.

该研究介绍了 TREC Fair Ranking 跟踪项目，实现了服务于不同内容提供商公平性和相关度的基准评估，发布了一个数据集，目标是要公正地呈现来自多个未知组的相关作者的重要性，重点关注能够在各种组别定义下表现出强大性能的系统的开发。