We present PULP-NN, an optimized computing library for a parallel
ultra-low-power tightly coupled cluster of RISC-V processors. The key
innovation in PULP-NN is a set of kernels for Quantized Neural Network (QNN)
inference, targeting byte and sub-byte data types, down to INT-1, tuned for the
recent trend toward aggressive quantization in deep neural network inference.
The proposed library exploits both the digital signal processing (DSP)
extensions available in the PULP RISC-V processors and the cluster's
parallelism, achieving up to 15.5 MACs/cycle on INT-8 and improving performance
by up to 63x with respect to a sequential implementation on a single RISC-V
core implementing the baseline RV32IMC ISA. Using PULP-NN, a CIFAR-10 network
on an octa-core cluster runs in 30x and 19.6x less clock cycles than the
current state-of-the-art ARM CMSIS-NN library, running on STM32L4 and STM32H7
MCUs, respectively. The proposed library, when running on GAP-8 processor,
outperforms by 36.8x and by 7.45x the execution on energy efficient MCUs such
as STM32L4 and high-end MCUs such as STM32H7 respectively, when operating at
the maximum frequency. The energy efficiency on GAP-8 is 14.1x higher than
STM32L4 and 39.5x higher than STM32H7, at the maximum efficiency operating
point.

PULP-NN 是一种优化的计算库，用于并行超低功耗 RISC-V 处理器集群，其关键创新在于量化神经网络推理的内核，可实现对字节和子字节数据类型的优化，利用 PULP RISC-V 处理器提供的数字信号处理扩展和集群并行性，可以达到高达 15.5 个 MACs / 周期的性能，提高了多倍的执行效率，特别是在嵌入式系统中的优化效果相当明显。