Despite the clear performance benefits of data augmentations, little is known about why they are so effective. In this paper, we disentangle several key mechanisms through which data augmentations operate. Establishing an exchange rate between augmented and additional real data, we find that in out-of-distribution testing scenarios, augmentations which yield samples that are diverse, but inconsistent with the data distribution can be even more valuable than additional training data. Moreover, we find that data augmentations which encourage invariances can be more valuable than invariance alone, especially on small and medium sized training sets. Following this observation, we show that augmentations induce additional stochasticity during training, effectively flattening the loss landscape.

本文旨在研究数据扩展的机制，通过评估增强数据与额外真实数据之间的交换率发现，多样而不一致的增强数据可以比额外的训练数据更有价值，在小和中型训练集上，鼓励不变性的数据扩展可以比不变性本身更有价值，并证明了增强数据在训练过程中引入额外的随机性，从而有效地平稳了损失函数。

数据增强价值的探究：关于规模定律、不变性和隐式正则化的调查