Autonomous driving requires an accurate representation of the environment. A
strategy toward high accuracy is to fuse data from several sensors. Learned
Bird's-Eye View (BEV) encoders can achieve this by mapping data from individual
sensors into one joint latent space. For cost-efficient camera-only systems,
this provides an effective mechanism to fuse data from multiple cameras with
different views. Accuracy can further be improved by aggregating sensor
information over time. This is especially important in monocular camera systems
to account for the lack of explicit depth and velocity measurements. Thereby,
the effectiveness of developed BEV encoders crucially depends on the operators
used to aggregate temporal information and on the used latent representation
spaces. We analyze BEV encoders proposed in the literature and compare their
effectiveness, quantifying the effects of aggregation operators and latent
representations. While most existing approaches aggregate temporal information
either in image or in BEV latent space, our analyses and performance
comparisons suggest that these latent representations exhibit complementary
strengths. Therefore, we develop a novel temporal BEV encoder, TempBEV, which
integrates aggregated temporal information from both latent spaces. We consider
subsequent image frames as stereo through time and leverage methods from
optical flow estimation for temporal stereo encoding. Empirical evaluation on
the NuScenes dataset shows a significant improvement by TempBEV over the
baseline for 3D object detection and BEV segmentation. The ablation uncovers a
strong synergy of joint temporal aggregation in the image and BEV latent space.
These results indicate the overall effectiveness of our approach and make a
strong case for aggregating temporal information in both image and BEV latent
spaces.

通过融合多个传感器数据，通过学习的鸟瞰图编码器将不同视角的相机数据映射到一个联合的潜在空间中，通过在时域中聚合传感器信息，特别是单眼相机系统中缺乏明确的深度和速度测量时，能够进一步提高准确性。在文献中分析了鸟瞰图编码器并比较了它们的有效性，量化了聚合操作和潜在表示的影响。因此，我们开发了一种新颖的时域鸟瞰图编码器 TempBEV，它整合了来自两个潜在空间的聚合时域信息，并通过光流估计方法进行时域立体编码。在 NuScenes 数据集上的实证评估表明，TempBEV 在三维物体检测和鸟瞰图分割方面相比基线有显著改进。消融分析揭示了图像和鸟瞰图潜在空间中联合时域聚合的强大协同效应。这些结果表明了我们方法的整体有效性，并强烈支持在图像和鸟瞰图潜在空间中聚合时域信息。