Miscalibration in deep learning refers to there is a discrepancy between the
predicted confidence and performance. This problem usually arises due to the
overfitting problem, which is characterized by learning everything presented in
the training set, resulting in overconfident predictions during testing.
Existing methods typically address overfitting and mitigate the miscalibration
by adding a maximum-entropy regularizer to the objective function. The
objective can be understood as seeking a model that fits the ground-truth
labels by increasing the confidence while also maximizing the entropy of
predicted probabilities by decreasing the confidence. However, previous methods
lack clear guidance on confidence adjustment, leading to conflicting objectives
(increasing but also decreasing confidence). Therefore, we introduce a method
called Dynamic Regularization (DReg), which aims to learn what should be
learned during training thereby circumventing the confidence adjusting
trade-off. At a high level, DReg aims to obtain a more reliable model capable
of acknowledging what it knows and does not know. Specifically, DReg
effectively fits the labels for in-distribution samples (samples that should be
learned) while applying regularization dynamically to samples beyond model
capabilities (e.g., outliers), thereby obtaining a robust calibrated model
especially on the samples beyond model capabilities. Both theoretical and
empirical analyses sufficiently demonstrate the superiority of DReg compared
with previous methods.

深度学习中的误校准问题是指预测的置信度与性能之间存在差异。我们提出一种叫做 Dynamic Regularization (DReg) 的方法，通过动态调整正则化以提高模型的可靠性，适应于超出模型能力范围的样本，在理论和实证分析中已证明其优于之前的方法。