Ranking schemes drive many real-world decisions, like, where to study, whom
to hire, what to buy, etc. Many of these decisions often come with high
consequences. For example, a university can be deemed less prestigious if not
featured in a top-k list, and consumers might not even explore products that do
not get recommended to buyers. At the heart of most of these decisions are
opaque ranking schemes, which dictate the ordering of data entities, but their
internal logic is inaccessible or proprietary. Drawing inferences about the
ranking differences is like a guessing game to the stakeholders, like, the
rankees (i.e., the entities who are ranked, like product companies) and the
decision-makers (i.e., who use the rankings, like buyers). In this paper, we
aim to enable transparency in ranking interpretation by using algorithmic
rankers that learn from available data and by enabling human reasoning about
the learned ranking differences using explainable AI (XAI) methods. To realize
this aim, we leverage the exploration-explanation paradigm of human-data
interaction to let human stakeholders explore subsets and groupings of complex
multi-attribute ranking data using visual explanations of model fit and
attribute influence on rankings. We realize this explanation paradigm for
transparent ranking interpretation in TRIVEA, a visual analytic system that is
fueled by: i) visualizations of model fit derived from algorithmic rankers that
learn the associations between attributes and rankings from available data and
ii) visual explanations derived from XAI methods that help abstract important
patterns, like, the relative influence of attributes in different ranking
ranges. Using TRIVEA, end users not trained in data science have the agency to
transparently reason about the global and local behavior of the rankings
without the need to open black-box ranking models and develop confidence in the
resulting attribute-based inferences. We demonstrate the efficacy of TRIVEA
using multiple usage scenarios and subjective feedback from researchers with
diverse domain expertise. Keywords: Visual Analytics, Learning-to-Rank,
Explainable ML, Ranking

使用算法排序和可解释的 AI 方法实现透明的排名解释，通过可视化解释模型拟合和属性对排名的影响来探索和解释复杂的多属性排名数据的子集和分组，在不了解黑盒排名模型的情况下，帮助具备数据科学训练的最终用户透明地推理排名的全局和局部行为。

TRIVEA：透明排名解释：黑盒算法排名的可视化解释

TRIVEA: Transparent Ranking Interpretation using Visual Explanation of  Black-Box Algorithmic Rankers

In the past few years, there has been much work on incorporating fairness
requirements into algorithmic rankers, with contributions coming from the data
management, algorithms, information retrieval, and recommender systems
communities. In this survey we give a systematic overview of this work,
offering a broad perspective that connects formalizations and algorithmic
approaches across subfields. An important contribution of our work is in
developing a common narrative around the value frameworks that motivate
specific fairness-enhancing interventions in ranking. This allows us to unify
the presentation of mitigation objectives and of algorithmic techniques to help
meet those objectives or identify trade-offs. In this survey, we describe four
classification frameworks for fairness-enhancing interventions, along which we
relate the technical methods surveyed in this paper, discuss evaluation
datasets, and present technical work on fairness in score-based ranking. Then,
we present methods that incorporate fairness in supervised learning, and also
give representative examples of recent work on fairness in recommendation and
matchmaking systems. We also discuss evaluation frameworks for fair score-based
ranking and fair learning-to-rank, and draw a set of recommendations for the
evaluation of fair ranking methods.

本文对过去几年来有关将公平性要求纳入算法排名器的工作进行了系统概述，提供了跨学科的算法形式化和方法连接的广阔视角，并针对公平性增强干预分类架构描述了四个分类框架，并讨论了公平排序评估数据集和技术工作。我们还讨论了公平分数排序和公平学习排序的评估框架，并得出了有关公平排名方法评估的一系列建议。