Yang et al. (2023) discovered that removing a mere 1% of training points can
often lead to the flipping of a prediction. Given the prevalence of noisy data
in machine learning models, we pose the question: can we also result in the
flipping of a test prediction by relabeling a small subset of the training data
before the model is trained? In this paper, utilizing the extended influence
function, we propose an efficient procedure for identifying and relabeling such
a subset, demonstrating consistent success. This mechanism serves multiple
purposes: (1) providing a complementary approach to challenge model predictions
by recovering potentially mislabeled training points; (2) evaluating model
resilience, as our research uncovers a significant relationship between the
subset's size and the ratio of noisy data in the training set; and (3) offering
insights into bias within the training set. To the best of our knowledge, this
work represents the first investigation into the problem of identifying and
relabeling the minimal training subset required to flip a given prediction.

该研究利用扩展影响函数提出了一种有效的识别和重新标记最小训练样本以翻转给定预测的过程，评估模型的韧性，并提供有关训练集内偏差的见解。

用最小的训练子集重新标记以改变预测结果

Relabel Minimal Training Subset to Flip a Prediction

We compare classification and regression tasks in an overparameterized linear
model with Gaussian features. On the one hand, we show that with sufficient
overparameterization all training points are support vectors: solutions
obtained by least-squares minimum-norm interpolation, typically used for
regression, are identical to those produced by the hard-margin support vector
machine (SVM) that minimizes the hinge loss, typically used for training
classifiers. On the other hand, we show that there exist regimes where these
interpolating solutions generalize well when evaluated by the 0-1 test loss
function, but do not generalize if evaluated by the square loss function, i.e.
they approach the null risk. Our results demonstrate the very different roles
and properties of loss functions used at the training phase (optimization) and
the testing phase (generalization).

这篇研究论文探讨了使用高斯特征的超参数化线性模型在分类和回归任务中的表现，发现当过度参数化时，所有训练点都成为支持向量，从而演化出一个法向量的模型，比起普通的最小二乘法和解决分类问题的 SVM，更适用于两者的情况，并讨论了在不同 loss function 下，训练（优化）和测试（归纳）階段使用的方法的不同作用和特性。

过参数化情况下的分类与回归：损失函数是否重要？

Classification vs regression in overparameterized regimes: Does the loss  function matter?

Influence functions estimate the effect of removing a training point on a
model without the need to retrain. They are based on a first-order Taylor
approximation that is guaranteed to be accurate for sufficiently small changes
to the model, and so are commonly used to study the effect of individual points
in large datasets. However, we often want to study the effects of large groups
of training points, e.g., to diagnose batch effects or apportion credit between
different data sources. Removing such large groups can result in significant
changes to the model. Are influence functions still accurate in this setting?
In this paper, we find that across many different types of groups and for a
range of real-world datasets, the predicted effect (using influence functions)
of a group correlates surprisingly well with its actual effect, even if the
absolute and relative errors are large. Our theoretical analysis shows that
such strong correlation arises only under certain settings and need not hold in
general, indicating that real-world datasets have particular properties that
allow the influence approximation to be accurate.

本文研究影响函数在大规模数据集下的准确性，并发现对于大多数数据集，影响函数的预测效果与实际效果存在显著相关性，尽管可能存在较大的误差。