As the backbone technology of machine learning, deep neural networks (DNNs)
have have quickly ascended to the spotlight. Running DNNs on
resource-constrained mobile devices is, however, by no means trivial, since it
incurs high performance and energy overhead. While offloading DNNs to the cloud
for execution suffers unpredictable performance, due to the uncontrolled long
wide-area network latency. To address these challenges, in this paper, we
propose Edgent, a collaborative and on-demand DNN co-inference framework with
device-edge synergy. Edgent pursues two design knobs: (1) DNN partitioning that
adaptively partitions DNN computation between device and edge, in order to
leverage hybrid computation resources in proximity for real-time DNN inference.
(2) DNN right-sizing that accelerates DNN inference through early-exit at a
proper intermediate DNN layer to further reduce the computation latency. The
prototype implementation and extensive evaluations based on Raspberry Pi
demonstrate Edgent's effectiveness in enabling on-demand low-latency edge
intelligence.

提出了 Edgent，这是一种协作和按需的 DNN 合作推理框架，其旨在通过在设备和边缘之间自适应地分配 DNN 计算来提高实时 DNN 推理的性能，并通过在适当的中间 DNN 层进行早期退出来加速 DNN 推理。