Label corruption, where training samples have incorrect labels, can
significantly degrade the performance of machine learning models. This
corruption often arises from non-expert labeling or adversarial attacks.
Acquiring large, perfectly labeled datasets is costly, and retraining large
models from scratch when a clean dataset becomes available is computationally
expensive. To address this challenge, we propose Post-Training Correction, a
new paradigm that adjusts model parameters after initial training to mitigate
label noise, eliminating the need for retraining. We introduce Verifix, a novel
Singular Value Decomposition (SVD) based algorithm that leverages a small,
verified dataset to correct the model weights using a single update. Verifix
uses SVD to estimate a Clean Activation Space and then projects the model's
weights onto this space to suppress activations corresponding to corrupted
data. We demonstrate Verifix's effectiveness on both synthetic and real-world
label noise. Experiments on the CIFAR dataset with 25% synthetic corruption
show 7.36% generalization improvements on average. Additionally, we observe
generalization improvements of up to 2.63% on naturally corrupted datasets like
WebVision1.0 and Clothing1M.

提出后训练校正新范式以解决标签污染问题，引入基于奇异值分解（Singular Value Decomposition，SVD）的 Verifix 算法以消除机器学习模型中的错误标签，并通过投影模型权重到清洁激活空间来抑制对应于损坏数据的激活，实验证明 Verifix 在合成和真实数据集上都能有效提升泛化性能。