We propose the unified BRAVO challenge to benchmark the reliability of semantic segmentation models under realistic perturbations and unknown out-of-distribution (OOD) scenarios. We define two categories of reliability: (1) semantic reliability, which reflects the model's accuracy and calibration when exposed to various perturbations; and (2) OOD reliability, which measures the model's ability to detect object classes that are unknown during training. The challenge attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal interesting insights into the importance of large-scale pre-training and minimal architectural design in developing robust and reliable semantic segmentation models.

本研究解决了语义分割模型在现实扰动和未知分布外场景下的可靠性评估问题。提出的统一BRAVO挑战明确了语义可靠性和分布外可靠性两类标准，结果揭示了大规模预训练和简约架构设计在提升模型鲁棒性与可靠性方面的重要性。

BRAVO语义分割挑战赛在UNCV2024中的结果