Generative text-to-image models enable us to synthesize unlimited amounts of
images in a controllable manner, spurring many recent efforts to train vision
models with synthetic data. However, every synthetic image ultimately
originates from the upstream data used to train the generator. What additional
value does the intermediate generator provide over directly training on
relevant parts of the upstream data? Grounding this question in the setting of
image classification, we compare finetuning on task-relevant, targeted
synthetic data generated by Stable Diffusion -- a generative model trained on
the LAION-2B dataset -- against finetuning on targeted real images retrieved
directly from LAION-2B. We show that while synthetic data can benefit some
downstream tasks, it is universally matched or outperformed by real data from
our simple retrieval baseline. Our analysis suggests that this underperformance
is partially due to generator artifacts and inaccurate task-relevant visual
details in the synthetic images. Overall, we argue that retrieval is a critical
baseline to consider when training with synthetic data -- a baseline that
current methods do not yet surpass. We release code, data, and models at
this https URL

通过对比采用生成模型生成的人工数据和来自真实数据的有针对性图像进行微调，在任务上针对性生成的合成数据被真实数据普遍匹配或超越，这表明合成图像中存在生成器伪像和不准确的任务相关视觉细节。总体而言，我们认为在使用合成数据进行训练时，检索是一个必须考虑的关键基线。