Recent research on robustness has revealed significant performance gaps
between neural image classifiers trained on datasets that are similar to the
test set, and those that are from a naturally shifted distribution, such as
sketches, paintings, and animations of the object categories observed during
training. Prior work focuses on reducing this gap by designing engineered
augmentations of training data or through unsupervised pretraining of a single
large model on massive in-the-wild training datasets scraped from the Internet.
However, the notion of a dataset is also undergoing a paradigm shift in recent
years. With drastic improvements in the quality, ease-of-use, and access to
modern generative models, generated data is pervading the web. In this light,
we study the question: How do these generated datasets influence the natural
robustness of image classifiers? We find that Imagenet classifiers trained on
real data augmented with generated data achieve higher accuracy and effective
robustness than standard training and popular augmentation strategies in the
presence of natural distribution shifts. We analyze various factors influencing
these results, including the choice of conditioning strategies and the amount
of generated data. Lastly, we introduce and analyze an evolving generated
dataset, ImageNet-G-v1, to better benchmark the design, utility, and critique
of standalone generated datasets for robust and trustworthy machine learning.
The code and datasets are available at
this https URL.

本文研究了生成数据集对图像分类器自然鲁棒性的影响，发现与标准训练和流行的数据增强策略相比，使用真实数据和生成数据相结合进行训练可以提高 Imagenet 分类器的准确性和鲁棒性，同时分析了不同因素对结果的影响，并介绍了 ImageNet-G-v1 数据集。