Edge intelligence enables resource-demanding Deep Neural Network (DNN)
inference without transferring original data, addressing concerns about data
privacy in consumer Internet of Things (IoT) devices. For privacy-sensitive
applications, deploying models in hardware-isolated trusted execution
environments (TEEs) becomes essential. However, the limited secure memory in
TEEs poses challenges for deploying DNN inference, and alternative techniques
like model partitioning and offloading introduce performance degradation and
security issues. In this paper, we present a novel approach for advanced model
deployment in TrustZone that ensures comprehensive privacy preservation during
model inference. We design a memory-efficient management method to support
memory-demanding inference in TEEs. By adjusting the memory priority, we
effectively mitigate memory leakage risks and memory overlap conflicts,
resulting in 32 lines of code alterations in the trusted operating system.
Additionally, we leverage two tiny libraries: S-Tinylib (2,538 LoCs), a tiny
deep learning library, and Tinylibm (827 LoCs), a tiny math library, to support
efficient inference in TEEs. We implemented a prototype on Raspberry Pi 3B+ and
evaluated it using three well-known lightweight DNN models. The experimental
results demonstrate that our design significantly improves inference speed by
3.13 times and reduces power consumption by over 66.5% compared to non-memory
optimization method in TEEs.

边缘智能允许资源密集型深度神经网络进行推理，而无需传输原始数据，解决了消费者物联网设备中的数据隐私问题。我们提出了一种在 TrustZone 中进行先进模型部署的新方法，确保在模型推理过程中综合保护隐私。我们设计了一种内存高效的管理方法，支持内存密集型推理，通过调整内存优先级，有效地减少内存泄漏风险和内存冲突，并在受信任的操作系统中进行了 32 行代码的修改。此外，我们利用了两个小型库：S-Tinylib（2,538 行代码），一个小型深度学习库，和 Tinylibm（827 行代码），一个小型数学库，以支持在受信任执行环境中的高效推理。我们在 Raspberry Pi 3B + 上实现了一个原型，并使用三个知名的轻量级 DNN 模型进行了评估。实验结果表明，与在 TEE 中使用非内存优化方法相比，我们的设计能够将推理速度提高 3.13 倍，并降低功耗超过 66.5%。