Autonomous vehicles (AV) require that neural networks used for perception be
robust to different viewpoints if they are to be deployed across many types of
vehicles without the repeated cost of data collection and labeling for each. AV
companies typically focus on collecting data from diverse scenarios and
locations, but not camera rig configurations, due to cost. As a result, only a
small number of rig variations exist across most fleets. In this paper, we
study how AV perception models are affected by changes in camera viewpoint and
propose a way to scale them across vehicle types without repeated data
collection and labeling. Using bird's eye view (BEV) segmentation as a
motivating task, we find through extensive experiments that existing perception
models are surprisingly sensitive to changes in camera viewpoint. When trained
with data from one camera rig, small changes to pitch, yaw, depth, or height of
the camera at inference time lead to large drops in performance. We introduce a
technique for novel view synthesis and use it to transform collected data to
the viewpoint of target rigs, allowing us to train BEV segmentation models for
diverse target rigs without any additional data collection or labeling cost. To
analyze the impact of viewpoint changes, we leverage synthetic data to mitigate
other gaps (content, ISP, etc). Our approach is then trained on real data and
evaluated on synthetic data, enabling evaluation on diverse target rigs. We
release all data for use in future work. Our method is able to recover an
average of 14.7% of the IoU that is otherwise lost when deploying to new rigs.

自动驾驶车辆需要神经网络在感知方面能够适应不同的视角，以便在多种类型的车辆中使用而无需重复地进行数据收集和标注。本研究通过大量实验发现，现有的感知模型对于摄像机视角的变化非常敏感，因此提出了一种方法来在车辆类型之间进行缩放，从而避免了额外的数据收集和标注成本，并通过引入新颖的视图合成技术，训练适用于各种车辆类型的鸟瞰图分割模型。