Predictive models may generate biased predictions when classifying imbalanced
datasets. This happens when the model favors the majority class, leading to low
performance in accurately predicting the minority class. To address this issue,
balancing or resampling methods are critical pre-processing steps in the
modeling process. However, there have been debates and questioning of the
functionality of these methods in recent years. In particular, many candidate
models may exhibit very similar predictive performance, which is called the
Rashomon effect, in model selection. Selecting one of them without considering
predictive multiplicity which is the case of yielding conflicting models'
predictions for any sample may lead to a loss of using another model. In this
study, in addition to the existing debates, the impact of balancing methods on
predictive multiplicity is examined through the Rashomon effect. It is
important because the blind model selection is risky from a set of
approximately equally accurate models. This may lead to serious problems in
model selection, validation, and explanation. To tackle this matter, we
conducted real dataset experiments to observe the impact of balancing methods
on predictive multiplicity through the Rashomon effect. Our findings showed
that balancing methods inflate the predictive multiplicity, and they yield
varying results. To monitor the trade-off between performance and predictive
multiplicity for conducting the modeling process responsibly, we proposed using
the extended performance-gain plot for the Rashomon effect.

通过 Rashomon 效应研究平衡方法对预测多样性的影响，发现平衡方法会增加预测多样性并产生不同结果。为了在建模过程中负责任地监控性能和预测多样性之间的权衡，我们提出使用扩展性能增益图进行模型选择。

不平衡分类中平衡方法的拉细幕效应实验研究

An Experimental Study on the Rashomon Effect of Balancing Methods in  Imbalanced Classification

Imbalanced data poses a significant challenge in classification as model
performance is affected by insufficient learning from minority classes.
Balancing methods are often used to address this problem. However, such
techniques can lead to problems such as overfitting or loss of information.
This study addresses a more challenging aspect of balancing methods - their
impact on model behavior. To capture these changes, Explainable Artificial
Intelligence tools are used to compare models trained on datasets before and
after balancing. In addition to the variable importance method, this study uses
the partial dependence profile and accumulated local effects techniques. Real
and simulated datasets are tested, and an open-source Python package edgaro is
developed to facilitate this analysis. The results obtained show significant
changes in model behavior due to balancing methods, which can lead to biased
models toward a balanced distribution. These findings confirm that balancing
analysis should go beyond model performance comparisons to achieve higher
reliability of machine learning models. Therefore, we propose a new method
performance gain plot for informed data balancing strategy to make an optimal
selection of balancing method by analyzing the measure of change in model
behavior versus performance gain.

本研究采用可解释的人工智能工具，探讨样本平衡方法对模型行为的影响，旨在选出一种最佳的数据平衡策略以提高机器学习模型的可靠性。研究结果表明，平衡方法的使用会显著改变模型行为，呈现出对平衡分布的偏见。