The emergence of the Internet of Things (IoT) has resulted in a remarkable
amount of data generated on edge devices, which are often processed using AI
algorithms. On-device learning enables edge platforms to continually adapt the
AI models to user personal data and further allows for a better service
quality. However, AI training on resource-limited devices is extremely
difficult because of the intensive computing workload and the significant
amount of on-chip memory consumption exacted by deep neural networks (DNNs). To
mitigate this, we propose to use embedded dynamic random-access memory (eDRAM)
as the main storage medium of training data. Compared with static random-access
memory (SRAM), eDRAM introduces more than $2\times$ improvement on storage
density, enabling reduced off-chip memory traffic. However, to keep the stored
data intact, eDRAM is required to perform the power-hungry data refresh
operations.
eDRAM refresh can be eliminated if the data is stored for a period of time
that is shorter than the eDRAM retention time. To achieve this, we design a
novel reversible DNN architecture that enables a significantly reduced data
lifetime during the training process and removes the need for eDRAM refresh. We
further design an efficient on-device training engine, termed~\textit{CAMEL},
that uses eDRAM as the main on-chip memory. CAMEL enables the intermediate
results during training to fit fully in on-chip eDRAM arrays and completely
eliminates the off-chip DRAM traffic during the training process. We evaluate
our CAMEL system on multiple DNNs with different datasets, demonstrating a more
than $3\times$ saving on total DNN training energy consumption than the other
baselines, while achieving a similar (even better) performance in validation
accuracy.

本研究提出了使用嵌入式动态随机访问存储器 (eDRAM) 作为训练数据的主要存储介质来解决物联网 (IoT) 边缘设备上的深度神经网络训练问题，并设计了一种新颖的可逆 DNN 架构和高效的设备端训练引擎 CAMEL 来实现不需 eDRAM 刷新的数据训练过程， 从而大大减少了 DNN 训练的能耗并在验证精度上实现了类似（甚至更好）的性能。