Much of the focus in the design of deep neural networks has been on improving accuracy, leading to more powerful yet highly complex network architectures that are difficult to deploy in practical scenarios, particularly on edge devices such as mobile and other consumer devices, given their high computational and memory requirements. As a result, there has been a recent interest in the design of quantitative metrics for evaluating deep neural networks that accounts for more than just model accuracy as the sole indicator of network performance. In this study, we continue the conversation towards universal metrics for evaluating the performance of deep neural networks for practical usage. In particular, we propose a new balanced metric called NetScore, which is designed specifically to provide a quantitative assessment of the balance between accuracy, computational complexity, and network architecture complexity of a deep neural network. In what is one of the largest comparative analysis between deep neural networks in literature, the NetScore metric, the top-1 accuracy metric, and the popular information density metric were compared across a diverse set of 50 different deep convolutional neural networks for image classification on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2012) dataset. The evaluation results across these three metrics for this diverse set of networks are presented in this study to act as a reference guide for practitioners in the field. The proposed NetScore metric, along with the other tested metrics, are by no means perfect, but the hope is to push the conversation towards better universal metrics for evaluating deep neural networks for use in practical scenarios to help guide practitioners in model design.

本研究提出了一种名为NetScore的平衡指标，用于评估深度神经网络在实际边缘设备上的性能，该指标综合考虑了准确性、计算复杂度和网络结构复杂度等因素。通过对60种不同的卷积神经网络在ImageNet上进行比较，本研究总结了这些网络在准确性、NetScore和信息密度等指标下的表现，以帮助实务人员在边缘场景下设计模型。

NetScore: 面向大规模深度神经网络实用边缘设备应用的通用性性能评估指标