Detecting a diverse range of objects under various driving scenarios is
essential for the effectiveness of autonomous driving systems. However, the
real-world data collected often lacks the necessary diversity presenting a
long-tail distribution. Although synthetic data has been utilized to overcome
this issue by generating virtual scenes, it faces hurdles such as a significant
domain gap and the substantial efforts required from 3D artists to create
realistic environments. To overcome these challenges, we present ARSim, a fully
automated, comprehensive, modular framework designed to enhance real multi-view
image data with 3D synthetic objects of interest. The proposed method
integrates domain adaptation and randomization strategies to address covariate
shift between real and simulated data by inferring essential domain attributes
from real data and employing simulation-based randomization for other
attributes. We construct a simplified virtual scene using real data and
strategically place 3D synthetic assets within it. Illumination is achieved by
estimating light distribution from multiple images capturing the surroundings
of the vehicle. Camera parameters from real data are employed to render
synthetic assets in each frame. The resulting augmented multi-view consistent
dataset is used to train a multi-camera perception network for autonomous
vehicles. Experimental results on various AV perception tasks demonstrate the
superior performance of networks trained on the augmented dataset.

为了增强自动驾驶系统的有效性，本文提出了 ARSim，一个全自动、综合和模块化框架，用于通过将 3D 合成对象与真实多视图图像数据相结合，来提高真实世界场景的多样性和覆盖范围。实验结果表明，在各种自动驾驶感知任务上，使用增强数据集训练的网络性能更优。