Although overparameterized models have shown their success on many machine
learning tasks, the accuracy could drop on the testing distribution that is
different from the training one. This accuracy drop still limits applying
machine learning in the wild. At the same time, importance weighting, a
traditional technique to handle distribution shifts, has been demonstrated to
have less or even no effect on overparameterized models both empirically and
theoretically. In this paper, we propose importance tempering to improve the
decision boundary and achieve consistently better results for overparameterized
models. Theoretically, we justify that the selection of group temperature can
be different under label shift and spurious correlation setting. At the same
time, we also prove that properly selected temperatures can extricate the
minority collapse for imbalanced classification. Empirically, we achieve
state-of-the-art results on worst group classification tasks using importance
tempering.

本文探讨了过度参数化模型的精度问题，提出了一种名为 “importance tempering” 的方法来解决分布偏移问题，并在实验中取得了最佳状态分类任务的最佳结果。