MixUp is a data augmentation strategy where additional samples are generated
during training by combining random pairs of training samples and their labels.
However, selecting random pairs is not potentially an optimal choice. In this
work, we propose TDMixUp, a novel MixUp strategy that leverages Training
Dynamics and allows more informative samples to be combined for generating new
data samples. Our proposed TDMixUp first measures confidence, variability,
(Swayamdipta et al., 2020), and Area Under the Margin (AUM) (Pleiss et al.,
2020) to identify the characteristics of training samples (e.g., as
easy-to-learn or ambiguous samples), and then interpolates these characterized
samples. We empirically validate that our method not only achieves competitive
performance using a smaller subset of the training data compared with strong
baselines, but also yields lower expected calibration error on the pre-trained
language model, BERT, on both in-domain and out-of-domain settings in a wide
range of NLP tasks. We publicly release our code.

本文提出 TDMixUp 方法，利用训练动态，基于易于学习或模糊样本测量置信度、可变性、AUM 特征，让更具备信息量的样本相互插值，生成新样本，实验证明该方法不仅在较小的训练数据子集上取得竞争性能，而且在广泛的自然语言处理任务中，使用预训练语言模型 BERT 取得较低的期望校准误差。我们公开发布了我们的代码。