Predictive multiplicity refers to the phenomenon in which classification
tasks may admit multiple competing models that achieve almost-equally-optimal
performance, yet generate conflicting outputs for individual samples. This
presents significant concerns, as it can potentially result in systemic
exclusion, inexplicable discrimination, and unfairness in practical
applications. Measuring and mitigating predictive multiplicity, however, is
computationally challenging due to the need to explore all such
almost-equally-optimal models, known as the Rashomon set, in potentially huge
hypothesis spaces. To address this challenge, we propose a novel framework that
utilizes dropout techniques for exploring models in the Rashomon set. We
provide rigorous theoretical derivations to connect the dropout parameters to
properties of the Rashomon set, and empirically evaluate our framework through
extensive experimentation. Numerical results show that our technique
consistently outperforms baselines in terms of the effectiveness of predictive
multiplicity metric estimation, with runtime speedup up to $20\times \sim
5000\times$. With efficient Rashomon set exploration and metric estimation,
mitigation of predictive multiplicity is then achieved through dropout ensemble
and model selection.

通过利用 dropout 技术来探索 Rashomon 集合中的模型，提出了一个用于衡量和减轻预测多样性的新框架，通过严格的理论推导和实验证明该技术在预测多样性度量估计方面始终优于基准方法，并通过 dropout 集成和模型选择实现了高效的 Rashomon 集合探索和度量估计。

基于 Dropout 的 Rashomon 集探索技术用于高效的预测乘性估计

Dropout-Based Rashomon Set Exploration for Efficient Predictive  Multiplicity Estimation

Prediction problems often admit competing models that perform almost equally
well. This effect challenges key assumptions in machine learning when competing
models assign conflicting predictions. In this paper, we define predictive
multiplicity as the ability of a prediction problem to admit competing models
with conflicting predictions. We introduce formal measures to evaluate the
severity of predictive multiplicity and develop integer programming tools to
compute them exactly for linear classification problems. We apply our tools to
measure predictive multiplicity in recidivism prediction problems. Our results
show that real-world datasets may admit competing models that assign wildly
conflicting predictions, and motivate the need to measure and report predictive
multiplicity in model development.

本研究旨在解决机器学习中不同模型之间预测结果冲突的问题，定义了预测重复性作为预测问题接受有多个竞争模型的能力，并引入形式化的度量方式和整数规划工具来计算它们，应用于反复犯罪预测问题中，结果显示现实世界的数据集会接受互相冲突的模型，提出了需要在模型评估中报告预测重复性的建议。