The network edge's role in Artificial Intelligence (AI) inference processing
is rapidly expanding, driven by a plethora of applications seeking
computational advantages. These applications strive for data-driven efficiency,
leveraging robust AI capabilities and prioritizing real-time responsiveness.
However, as demand grows, so does system complexity. The proliferation of AI
inference accelerators showcases innovation but also underscores challenges,
particularly the varied software and hardware configurations of these devices.
This diversity, while advantageous for certain tasks, introduces hurdles in
device integration and coordination. In this paper, our objectives are
three-fold. Firstly, we outline the requirements and components of a framework
that accommodates hardware diversity. Next, we assess the impact of device
heterogeneity on AI inference performance, identifying strategies to optimize
outcomes without compromising service quality. Lastly, we shed light on the
prevailing challenges and opportunities in this domain, offering insights for
both the research community and industry stakeholders.

网络边缘在人工智能（AI）推理处理中的作用正在快速扩大，驱动力是众多寻求计算优势的应用程序。