Accurate obstacle identification represents a fundamental challenge within
the scope of near-field perception for autonomous driving. Conventionally,
fisheye cameras are frequently employed for comprehensive surround-view
perception, including rear-view obstacle localization. However, the performance
of such cameras can significantly deteriorate in low-light conditions, during
nighttime, or when subjected to intense sun glare. Conversely, cost-effective
sensors like ultrasonic sensors remain largely unaffected under these
conditions. Therefore, we present, to our knowledge, the first end-to-end
multimodal fusion model tailored for efficient obstacle perception in a
bird's-eye-view (BEV) perspective, utilizing fisheye cameras and ultrasonic
sensors. Initially, ResNeXt-50 is employed as a set of unimodal encoders to
extract features specific to each modality. Subsequently, the feature space
associated with the visible spectrum undergoes transformation into BEV. The
fusion of these two modalities is facilitated via concatenation. At the same
time, the ultrasonic spectrum-based unimodal feature maps pass through
content-aware dilated convolution, applied to mitigate the sensor misalignment
between two sensors in the fused feature space. Finally, the fused features are
utilized by a two-stage semantic occupancy decoder to generate grid-wise
predictions for precise obstacle perception. We conduct a systematic
investigation to determine the optimal strategy for multimodal fusion of both
sensors. We provide insights into our dataset creation procedures, annotation
guidelines, and perform a thorough data analysis to ensure adequate coverage of
all scenarios. When applied to our dataset, the experimental results underscore
the robustness and effectiveness of our proposed multimodal fusion approach.

基于鱼眼相机和超声波传感器的多模态融合模型，实现了高效障碍物感知，通过 BEV 视角提取特征并融合两种传感器的特征空间，通过两阶段语义占用解码器生成准确的障碍物感知预测。