The applications of artificial intelligence (AI) are rapidly evolving, and
they are also commonly used in safety-critical domains, such as autonomous
driving and medical diagnosis, where functional safety is paramount. In
AI-driven systems, uncertainty estimation allows the user to avoid
overconfidence predictions and achieve functional safety. Therefore, the
robustness and reliability of model predictions can be improved. However,
conventional uncertainty estimation methods, such as the deep ensemble method,
impose high computation and, accordingly, hardware (latency and energy)
overhead because they require the storage and processing of multiple models.
Alternatively, Monte Carlo dropout (MC-dropout) methods, although having low
memory overhead, necessitate numerous ($\sim 100$) forward passes, leading to
high computational overhead and latency. Thus, these approaches are not
suitable for battery-powered edge devices with limited computing and memory
resources. In this paper, we propose the Tiny-Deep Ensemble approach, a
low-cost approach for uncertainty estimation on edge devices. In our approach,
only normalization layers are ensembled $M$ times, with all ensemble members
sharing common weights and biases, leading to a significant decrease in storage
requirements and latency. Moreover, our approach requires only one forward pass
in a hardware architecture that allows batch processing for inference and
uncertainty estimation. Furthermore, it has approximately the same memory
overhead compared to a single model. Therefore, latency and memory overhead are
reduced by a factor of up to $\sim M\times$. Nevertheless, our method does not
compromise accuracy, with an increase in inference accuracy of up to $\sim 1\%$
and a reduction in RMSE of $17.17\%$ in various benchmark datasets, tasks, and
state-of-the-art architectures.

本文提出了一种适用于边缘设备的低成本方法 ——Tiny-Deep Ensemble 方法，用于不确定性估计。该方法通过进行多次归一化层集成而不需要额外的存储空间和延迟，从而显著降低了存储需求和延迟，同时保持了与单模型相近的内存开销，并在各种基准数据集、任务和最先进的架构中提高了推理准确性。