We introduce Deep Augmentation, an approach to data augmentation using dropout to dynamically transform a targeted layer within a neural network, with the option to use the stop-gradient operation, offering significant improvements in model performance and generalization. We demonstrate the efficacy of Deep Augmentation through extensive experiments on contrastive learning tasks in computer vision and NLP domains, where we observe substantial performance gains with ResNets and Transformers as the underlying models. Our experimentation reveals that targeting deeper layers with Deep Augmentation outperforms augmenting the input data, and the simple network- and data-agnostic nature of this approach enables its seamless integration into computer vision and NLP pipelines.

采用“Deep Augmentation”方法，通过在神经网络内部针对性地应用dropout实现数据增广，并采用stop-gradient操作进一步提高模型性能和泛化能力，在视觉和自然语言处理领域的对比学习任务中通过对ResNets和Transformers模型的广泛实验证明效果显著，与增广输入数据相比，针对深层模型的Deep Augmentation效果更好，而且该方法简单且对网络和数据无限制，方便集成到计算机视觉和自然语言处理流程中。

在高激活空间中通过变换增强自监督学习的深度增强