Mixup, which creates synthetic training instances by linearly interpolating
random sample pairs, is a simple and yet effective regularization technique to
boost the performance of deep models trained with SGD. In this work, we report
a previously unobserved phenomenon in Mixup training: on a number of standard
datasets, the performance of Mixup-trained models starts to decay after
training for a large number of epochs, giving rise to a U-shaped generalization
curve. This behavior is further aggravated when the size of original dataset is
reduced. To help understand such a behavior of Mixup, we show theoretically
that Mixup training may introduce undesired data-dependent label noises to the
synthesized data. Via analyzing a least-square regression problem with a random
feature model, we explain why noisy labels may cause the U-shaped curve to
occur: Mixup improves generalization through fitting the clean patterns at the
early training stage, but as training progresses, Mixup becomes over-fitting to
the noise in the synthetic data. Extensive experiments are performed on a
variety of benchmark datasets, validating this explanation.

通过线性插值的方法创建合成训练样例的 Mixup 是优化 SGD 的深度模型的简单而有效的正则化技术，本文显示 Mixup 训练中的 U 形泛化曲线现象， 通过一系列数据集的实验证明了混入嘈杂标签会让 Mixup 在合成数据中出现 U 形泛化曲线现象的理论解释。