Recently, a number of image-mixing-based augmentation techniques have been
introduced to improve the generalization of deep neural networks. In these
techniques, two or more randomly selected natural images are mixed together to
generate an augmented image. Such methods may not only omit important portions
of the input images but also introduce label ambiguities by mixing images
across labels resulting in misleading supervisory signals. To address these
limitations, we propose DiffuseMix, a novel data augmentation technique that
leverages a diffusion model to reshape training images, supervised by our
bespoke conditional prompts. First, concatenation of a partial natural image
and its generated counterpart is obtained which helps in avoiding the
generation of unrealistic images or label ambiguities. Then, to enhance
resilience against adversarial attacks and improves safety measures, a randomly
selected structural pattern from a set of fractal images is blended into the
concatenated image to form the final augmented image for training. Our
empirical results on seven different datasets reveal that DiffuseMix achieves
superior performance compared to existing state-of the-art methods on tasks
including general classification,fine-grained classification, fine-tuning, data
scarcity, and adversarial robustness. Augmented datasets and codes are
available here: this https URL

最近，引入了一些基于图像混合的数据增强技术来改善深度神经网络的泛化能力。我们提出了一种名为 DiffuseMix 的新型数据增强技术，该技术利用扩散模型重塑训练图像，并使用我们量身定制的条件提示进行监督。DiffuseMix 在包括一般分类、细粒度分类、微调、数据稀缺性和对抗鲁棒性等任务上相较于现有最先进的方法取得了优越性能。