Pre-trained language models (PLMs) achieve remarkable performance on many
downstream tasks, but may fail in giving reliable estimates of their predictive
uncertainty. Given the lack of a comprehensive understanding of PLMs
calibration, we take a close look into this new research problem, aiming to
answer two questions: (1) Do PLMs learn to become calibrated in the training
process? (2) How effective are existing calibration methods? For the first
question, we conduct fine-grained control experiments to study the dynamic
change in PLMs' calibration performance in training. We consider six factors as
control variables, including dataset difficulty, available training samples,
training steps, the number of tunable parameters, model scale, and pretraining.
In experiments, we observe a consistent change in calibration performance
across six factors. We find that PLMs don't learn to become calibrated in
training, evidenced by the continual increase in confidence, no matter the
predictions are correct or not. We highlight that our finding presents some
contradiction with two established conclusions: (a) Larger PLMs are more
calibrated; (b) Pretraining improves model calibration. Next, we study the
effectiveness of existing calibration methods in mitigating the overconfidence
issue, in both in-distribution and various out-of-distribution settings.
Besides unlearnable calibration methods, we adapt two recently proposed
learnable methods that directly collect data to train models to have reasonable
confidence estimations. Also, we propose extended learnable methods based on
existing ones to further improve or maintain PLMs calibration without
sacrificing the original task performance. Experimental results show that
learnable methods significantly reduce PLMs' confidence in wrong predictions,
and our methods exhibit superior performance compared with previous methods.

本文通过精细控制实验和给定数据集，考察了 PLM 模型在训练过程中的校准性能，发现仍然存在校准问题；然后对比了多种方法，提出基于已有方法的扩展性学习方法，证明其显著提高模型的不确定性估计水平。