Precise situational awareness is required for the safe decision-making of
assisted and automated driving (AAD) functions. Panoptic segmentation is a
promising perception technique to identify and categorise objects, impending
hazards, and driveable space at a pixel level. While segmentation quality is
generally associated with the quality of the camera data, a comprehensive
understanding and modelling of this relationship are paramount for AAD system
designers. Motivated by such a need, this work proposes a unifying pipeline to
assess the robustness of panoptic segmentation models for AAD, correlating it
with traditional image quality. The first step of the proposed pipeline
involves generating degraded camera data that reflects real-world noise
factors. To this end, 19 noise factors have been identified and implemented
with 3 severity levels. Of these factors, this work proposes novel models for
unfavourable light and snow. After applying the degradation models, three
state-of-the-art CNN- and vision transformers (ViT)-based panoptic segmentation
networks are used to analyse their robustness. The variations of the
segmentation performance are then correlated to 8 selected image quality
metrics. This research reveals that: 1) certain specific noise factors produce
the highest impact on panoptic segmentation, i.e. droplets on lens and Gaussian
noise; 2) the ViT-based panoptic segmentation backbones show better robustness
to the considered noise factors; 3) some image quality metrics (i.e. LPIPS and
CW-SSIM) correlate strongly with panoptic segmentation performance and
therefore they can be used as predictive metrics for network performance.

准确的情景感知对于辅助和自动驾驶功能 (AAD) 的安全决策至关重要。全景分割是一种有前景的感知技术，可以在像素级别上识别和分类对象、潜在危险和可驾驶空间。本研究提出了一个统一的流程，评估全景分割模型在 AAD 中的鲁棒性，并将其与传统图像质量进行相关联。