In the context of deep learning research, where model introductions
continually occur, the need for effective and efficient evaluation remains
paramount. Existing methods often emphasize accuracy metrics, overlooking
stability. To address this, the paper introduces the Accuracy-Stability Index
(ASI), a quantitative measure incorporating both accuracy and stability for
assessing deep learning models. Experimental results demonstrate the
application of ASI, and a 3D surface model is presented for visualizing ASI,
mean accuracy, and coefficient of variation. This paper addresses the important
issue of quantitative benchmarking metrics for deep learning models, providing
a new approach for accurately evaluating accuracy and stability of deep
learning models. The paper concludes with discussions on potential weaknesses
and outlines future research directions.

在深度学习研究的背景下，本文介绍了一个新的量化测量方法 —— 精确度稳定性指标（ASI），用于评估深度学习模型的精确度和稳定性。该方法结合了准确度和稳定性，并通过实验结果展示了 ASI 的应用，并提出了一个三维曲面模型，用于可视化 ASI、平均准确度和变异系数。本文解决了深度学习模型定量基准评估指标的重要问题，为准确评估深度学习模型的精确度和稳定性提供了一种新方法。文中还讨论了潜在的缺点，并概述了未来的研究方向。

ASI：用于评估深度学习模型的准确性稳定性指数

ASI: Accuracy-Stability Index for Evaluating Deep Learning Models

Deep neural networks (DNNs) are currently widely used for many artificial
intelligence (AI) applications including computer vision, speech recognition,
and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it
comes at the cost of high computational complexity. Accordingly, techniques
that enable efficient processing of DNNs to improve energy efficiency and
throughput without sacrificing application accuracy or increasing hardware cost
are critical to the wide deployment of DNNs in AI systems.
This article aims to provide a comprehensive tutorial and survey about the
recent advances towards the goal of enabling efficient processing of DNNs.
Specifically, it will provide an overview of DNNs, discuss various hardware
platforms and architectures that support DNNs, and highlight key trends in
reducing the computation cost of DNNs either solely via hardware design changes
or via joint hardware design and DNN algorithm changes. It will also summarize
various development resources that enable researchers and practitioners to
quickly get started in this field, and highlight important benchmarking metrics
and design considerations that should be used for evaluating the rapidly
growing number of DNN hardware designs, optionally including algorithmic
co-designs, being proposed in academia and industry.
The reader will take away the following concepts from this article:
understand the key design considerations for DNNs; be able to evaluate
different DNN hardware implementations with benchmarks and comparison metrics;
understand the trade-offs between various hardware architectures and platforms;
be able to evaluate the utility of various DNN design techniques for efficient
processing; and understand recent implementation trends and opportunities.

本文综述了目前深度神经网络的应用及其在计算机视觉、语音识别、机器人等 AI 任务中的高精度表现，同时也阐述了深度神经网络大规模部署所面临的高算力成本和能效问题，并提出了硬件设计、算法优化等多种解决方案以提高能效和算力，最后对比了各种深度神经网络的设计指标并展示了很多发展资源。