In this study, we introduce NeuralMatrix, a novel framework that enables the
computation of versatile deep neural networks (DNNs) on a single general matrix
multiplication (GEMM) accelerator. The proposed approach overcomes the
specificity limitations of ASIC-based accelerators while achieving
application-specific acceleration levels compared to general-purpose processors
such as CPUs and GPUs. We address the challenges of mapping both linear and
nonlinear operations in DNN computation to general matrix multiplications and
the impact of using a GEMM accelerator on DNN inference accuracy. Extensive
experiments are conducted on various DNN models from three popular categories
(i.e., CNN, Transformers, and GNN) as illustrative backbone models. Our results
demonstrate that DNNs suffer only up to a 2.02% accuracy loss after being
converted to general matrix multiplication, while achieving 113x to 19.44x
improvements in throughput per power compared to CPUs and GPUs.

本研究提出了一种新的框架 NeuralMatrix，可以在一个单一的通用矩阵乘法加速器上计算多功能的深度神经网络，同时在性能和特定应用的加速水平方面与 CPU 和 GPU 等通用处理器相比具有优势。