The Rashomon set is the set of models that perform approximately equally well
on a given dataset, and the Rashomon ratio is the fraction of all models in a
given hypothesis space that are in the Rashomon set. Rashomon ratios are often
large for tabular datasets in criminal justice, healthcare, lending, education,
and in other areas, which has practical implications about whether simpler
models can attain the same level of accuracy as more complex models. An open
question is why Rashomon ratios often tend to be large. In this work, we
propose and study a mechanism of the data generation process, coupled with
choices usually made by the analyst during the learning process, that
determines the size of the Rashomon ratio. Specifically, we demonstrate that
noisier datasets lead to larger Rashomon ratios through the way that
practitioners train models. Additionally, we introduce a measure called pattern
diversity, which captures the average difference in predictions between
distinct classification patterns in the Rashomon set, and motivate why it tends
to increase with label noise. Our results explain a key aspect of why simpler
models often tend to perform as well as black box models on complex, noisier
datasets.

在本研究中，我们探讨了数据生成过程中的机制以及分析师在学习过程中做出的选择，以确定 Rashomon 比率的大小。我们证明了更嘈杂的数据集通过从业者训练模型的方式导致较大的 Rashomon 比率。此外，我们还引入了一种称为模式多样性的度量，该度量捕捉了 Rashomon 集中不同分类模式之间的平均预测差异，并解释了它为什么会随着标签噪声的增加而增加。我们的结果解释了为什么简单模型在复杂且有噪声的数据集上通常能和黑盒子模型表现一样好的一个关键因素。