We present a conceptual framework, datamodeling, for analyzing the behavior
of a model class in terms of the training data. For any fixed "target" example
$x$, training set $S$, and learning algorithm, a datamodel is a parameterized
function $2^S \to \mathbb{R}$ that for any subset of $S' \subset S$ -- using
only information about which examples of $S$ are contained in $S'$ -- predicts
the outcome of training a model on $S'$ and evaluating on $x$. Despite the
potential complexity of the underlying process being approximated (e.g.,
end-to-end training and evaluation of deep neural networks), we show that even
simple linear datamodels can successfully predict model outputs. We then
demonstrate that datamodels give rise to a variety of applications, such as:
accurately predicting the effect of dataset counterfactuals; identifying
brittle predictions; finding semantically similar examples; quantifying
train-test leakage; and embedding data into a well-behaved and feature-rich
representation space. Data for this paper (including pre-computed datamodels as
well as raw predictions from four million trained deep neural networks) is
available at this https URL .

提出一种数据建模框架， 可以分析训练数据集对模型行为的影响，通过建立参数化函数，预测数据子集对于训练后的模型的影响并导出各种机器学习应用。