Data Shapley provides a principled approach to data valuation and plays a
crucial role in data-centric machine learning (ML) research. Data selection is
considered a standard application of Data Shapley. However, its data selection
performance has shown to be inconsistent across settings in the literature.
This study aims to deepen our understanding of this phenomenon. We introduce a
hypothesis testing framework and show that Data Shapley's performance can be no
better than random selection without specific constraints on utility functions.
We identify a class of utility functions, monotonically transformed modular
functions, within which Data Shapley optimally selects data. Based on this
insight, we propose a heuristic for predicting Data Shapley's effectiveness in
data selection tasks. Our experiments corroborate these findings, adding new
insights into when Data Shapley may or may not succeed.

通过引入一种假设检验框架并证明在没有对效用函数施加特定约束的情况下，Data Shapley 的性能不能比随机选择更好，我们深入研究了 Data Shapley 的性能差异。我们确定了一类效用函数，即单调转换模块化函数，在其中 Data Shapley 可以最优地选择数据，基于此，我们提出了一种启发式方法用于预测 Data Shapley 在数据选择任务中的有效性。我们的实验验证了这些发现，并进一步揭示了 Data Shapley 成功与否的一些新的洞察。