Deep learning models are extensively used in various safety critical
applications. Hence these models along with being accurate need to be highly
reliable. One way of achieving this is by quantifying uncertainty. Bayesian
methods for UQ have been extensively studied for Deep Learning models applied
on images but have been less explored for 3D modalities such as point clouds
often used for Robots and Autonomous Systems. In this work, we evaluate three
uncertainty quantification methods namely Deep Ensembles, MC-Dropout and
MC-DropConnect on the DarkNet21Seg 3D semantic segmentation model and
comprehensively analyze the impact of various parameters such as number of
models in ensembles or forward passes, and drop probability values, on task
performance and uncertainty estimate quality. We find that Deep Ensembles
outperforms other methods in both performance and uncertainty metrics. Deep
ensembles outperform other methods by a margin of 2.4% in terms of mIOU, 1.3%
in terms of accuracy, while providing reliable uncertainty for decision making.

本研究评估了三种不确定性量化方法，即深度集成，MC-Dropout 和 MC-DropConnect 在 DarkNet21Seg 3D 语义分割模型上的影响，并全面分析了各种参数对任务性能和不确定性估计质量的影响。研究发现，深度集成的表现在性能和不确定性度量方面优于其他方法，mIOU 提高了 2.4％，准确度提高了 1.3％，并提供可靠的不确定性来做出决策。