This paper explores the role and challenges of Artificial Intelligence (AI)
algorithms, specifically AI-based software elements, in autonomous driving
systems. These AI systems are fundamental in executing real-time critical
functions in complex and high-dimensional environments. They handle vital tasks
like multi-modal perception, cognition, and decision-making tasks such as
motion planning, lane keeping, and emergency braking. A primary concern relates
to the ability (and necessity) of AI models to generalize beyond their initial
training data. This generalization issue becomes evident in real-time
scenarios, where models frequently encounter inputs not represented in their
training or validation data. In such cases, AI systems must still function
effectively despite facing distributional or domain shifts. This paper
investigates the risk associated with overconfident AI models in
safety-critical applications like autonomous driving. To mitigate these risks,
methods for training AI models that help maintain performance without
overconfidence are proposed. This involves implementing certainty reporting
architectures and ensuring diverse training data. While various
distribution-based methods exist to provide safety mechanisms for AI models,
there is a noted lack of systematic assessment of these methods, especially in
the context of safety-critical automotive applications. Many methods in the
literature do not adapt well to the quick response times required in
safety-critical edge applications. This paper reviews these methods, discusses
their suitability for safety-critical applications, and highlights their
strengths and limitations. The paper also proposes potential improvements to
enhance the safety and reliability of AI algorithms in autonomous vehicles in
the context of rapid and accurate decision-making processes.

通过研究自动驾驶系统中基于人工智能的算法，本文探讨了人工智能模型的作用和挑战，特别是在复杂和高维环境中执行实时关键功能的软件元素。针对安全关键应用，为了减少 AI 模型过于自信带来的风险，提出了一些培训方法，如实现确定性报告架构和确保多样性训练数据。此外，还讨论了现有方法在安全关键应用中的适用性，强调了其优点和局限性，并提出了提高自动驾驶车辆中人工智能算法安全性和可靠性的潜在改进措施。