There is a growing demand for shifting the delivery of AI capability from
data centers on the cloud to edge or end devices, exemplified by the fast
emerging real-time AI-based apps running on smartphones, AR/VR devices,
autonomous vehicles, and various IoT devices. The shift has however been
seriously hampered by the large growing gap between DNN computing demands and
the computing power on edge or end devices. This article presents the design of
XGen, an optimizing framework for DNN designed to bridge the gap. XGen takes
cross-cutting co-design as its first-order consideration. Its full-stack
AI-oriented optimizations consist of a number of innovative optimizations at
every layer of the DNN software stack, all designed in a cooperative manner.
The unique technology makes XGen able to optimize various DNNs, including those
with an extreme depth (e.g., BERT, GPT, other transformers), and generate code
that runs several times faster than those from existing DNN frameworks, while
delivering the same level of accuracy.

XGen 是一个优化 DNN 的框架，采用横向协作设计，侧重于在边缘设备上运行实时 AI 应用程序，其优化包括创新的全栈 AI 优化，能够优化各种深度学习神经网络，生成比现有 DNN 框架更快、精度相同的代码。