As Neural Processing Units (NPU) or accelerators are increasingly deployed in
a variety of applications including safety critical applications such as
autonomous vehicle, and medical imaging, it is critical to understand the
fault-tolerance nature of the NPUs. We present a reliability study of Arm's
Ethos-U55, an important industrial-scale NPU being utilised in embedded and IoT
applications. We perform large scale RTL-level fault injections to characterize
Ethos-U55 against the Automotive Safety Integrity Level D (ASIL-D) resiliency
standard commonly used for safety-critical applications such as autonomous
vehicles. We show that, under soft errors, all four configurations of the NPU
fall short of the required level of resiliency for a variety of neural networks
running on the NPU. We show that it is possible to meet the ASIL-D level
resiliency without resorting to conventional strategies like Dual Core Lock
Step (DCLS) that has an area overhead of 100%. We achieve so through selective
protection, where hardware structures are selectively protected (e.g.,
duplicated, hardened) based on their sensitivity to soft errors and their
silicon areas. To identify the optimal configuration that minimizes the area
overhead while meeting the ASIL-D standard, the main challenge is the large
search space associated with the time-consuming RTL simulation. To address this
challenge, we present a statistical analysis tool that is validated against Arm
silicon and that allows us to quickly navigate hundreds of billions of fault
sites without exhaustive RTL fault injections. We show that by carefully
duplicating a small fraction of the functional blocks and hardening the Flops
in other blocks meets the ASIL-D safety standard while introducing an area
overhead of only 38%.

通过大规模的 RTL 级故障注入以及选择性保护硬件结构，本研究对 Arm 的 Ethos-U55 进行可靠性研究，展示了满足 ASIL-D 安全标准的优化配置，并在仅增加 38% 面积开销的情况下实现安全性。

对 Arm Ethos-U55 嵌入式机器学习加速器软错误容忍性的表征

Characterizing Soft-Error Resiliency in Arm's Ethos-U55 Embedded Machine  Learning Accelerator

On-device inference of machine learning models for mobile phones is desirable
due to its lower latency and increased privacy. Running such a
compute-intensive task solely on the mobile CPU, however, can be difficult due
to limited computing power, thermal constraints, and energy consumption. App
developers and researchers have begun exploiting hardware accelerators to
overcome these challenges. Recently, device manufacturers are adding neural
processing units into high-end phones for on-device inference, but these
account for only a small fraction of hand-held devices. In this paper, we
present how we leverage the mobile GPU, a ubiquitous hardware accelerator on
virtually every phone, to run inference of deep neural networks in real-time
for both Android and iOS devices. By describing our architecture, we also
discuss how to design networks that are mobile GPU-friendly. Our
state-of-the-art mobile GPU inference engine is integrated into the open-source
project TensorFlow Lite and publicly available at this https URL

本文介绍了如何利用手机上普遍存在的 GPU 加速器，在 Android 和 iOS 设备上实现深度神经网络的实时推断，并将其集成到开源项目 TensorFlow Lite 中。