Deep learning in computer vision has achieved great success with the price of
large-scale labeled training data. However, exhaustive data annotation is
impracticable for each task of all domains of interest, due to high labor costs
and unguaranteed labeling accuracy. Besides, the uncontrollable data collection
process produces non-IID training and test data, where undesired duplication
may exist. All these nuisances may hinder the verification of typical theories
and exposure to new findings. To circumvent them, an alternative is to generate
synthetic data via 3D rendering with domain randomization. We in this work push
forward along this line by doing profound and extensive research on bare
supervised learning and downstream domain adaptation. Specifically, under the
well-controlled, IID data setting enabled by 3D rendering, we systematically
verify the typical, important learning insights, e.g., shortcut learning, and
discover the new laws of various data regimes and network architectures in
generalization. We further investigate the effect of image formation factors on
generalization, e.g., object scale, material texture, illumination, camera
viewpoint, and background in a 3D scene. Moreover, we use the
simulation-to-reality adaptation as a downstream task for comparing the
transferability between synthetic and real data when used for pre-training,
which demonstrates that synthetic data pre-training is also promising to
improve real test results. Lastly, to promote future research, we develop a new
large-scale synthetic-to-real benchmark for image classification, termed S2RDA,
which provides more significant challenges for transfer from simulation to
reality. The code and datasets are available at
this https URL

本论文介绍了一种使用 3D 渲染和域随机化技术生成合成数据的方法，探究了深度学习和领域适应的相关理论和新发现，并利用模拟到现实的适应性作为下行任务，证明了合成数据预训练也有助于提高实际测试结果。最后，作者们发展了一个新的大规模合成到实际（S2RDA）图像分类基准，提供了更具挑战性的从模拟到现实的转移。