Recurrent Neural Networks (RNNs) are becoming increasingly important for time
series-related applications which require efficient and real-time
implementations. The two major types are Long Short-Term Memory (LSTM) and
Gated Recurrent Unit (GRU) networks. It is a challenging task to have
real-time, efficient, and accurate hardware RNN implementations because of the
high sensitivity to imprecision accumulation and the requirement of special
activation function implementations.
A key limitation of the prior works is the lack of a systematic design
optimization framework of RNN model and hardware implementations, especially
when the block size (or compression ratio) should be jointly optimized with RNN
type, layer size, etc. In this paper, we adopt the block-circulant matrix-based
framework, and present the Efficient RNN (E-RNN) framework for FPGA
implementations of the Automatic Speech Recognition (ASR) application. The
overall goal is to improve performance/energy efficiency under accuracy
requirement. We use the alternating direction method of multipliers (ADMM)
technique for more accurate block-circulant training, and present two design
explorations providing guidance on block size and reducing RNN training trials.
Based on the two observations, we decompose E-RNN in two phases: Phase I on
determining RNN model to reduce computation and storage subject to accuracy
requirement, and Phase II on hardware implementations given RNN model,
including processing element design/optimization, quantization, activation
implementation, etc. Experimental results on actual FPGA deployments show that
E-RNN achieves a maximum energy efficiency improvement of 37.4$\times$ compared
with ESE, and more than 2$\times$ compared with C-LSTM, under the same
accuracy.

本文提出了一种基于块循环矩阵框架的高效 RNN（E-RNN）框架，用于 FPGA 实现自动语音识别（ASR），旨在提高性能 / 能量效率，保证精度。采用了交替方向乘法器（ADMM）技术和两个设计探索来提高块循环训练的准确性和引导块大小选择，并将 E-RNN 分解为两个阶段：确定 RNN 模型以降低计算和存储，并进行硬件实现。在实际的 FPGA 部署中，实验结果表明，相对于 ESE 的最大能量效率提高了 37.4 倍，相对于 C-LSTM 提高了 2 倍以上，但保持了相同的准确性水平。