Orthogonal recurrent neural networks (ORNNs) are an appealing option for
learning tasks involving time series with long-term dependencies, thanks to
their simplicity and computational stability. However, these networks often
require a substantial number of parameters to perform well, which can be
prohibitive in power-constrained environments, such as compact devices. One
approach to address this issue is neural network quantization. The construction
of such networks remains an open problem, acknowledged for its inherent
instability.In this paper, we explore the quantization of the recurrent and
input weight matrices in ORNNs, leading to Quantized approximately Orthogonal
RNNs (QORNNs). We investigate one post-training quantization (PTQ) strategy and
three quantization-aware training (QAT) algorithms that incorporate orthogonal
constraints and quantized weights. Empirical results demonstrate the advantages
of employing QAT over PTQ. The most efficient model achieves results similar to
state-of-the-art full-precision ORNN and LSTM on a variety of standard
benchmarks, even with 3-bits quantization.

针对计算能力有限的设备，本文研究了正交循环神经网络（ORNNs）的量化问题，并提出了通过后训练量化和量化感知训练方法来实现量化正交 RNNs（QORNNs），实验结果表明量化感知训练比后训练量化更有效，最高效的模型在各种标准测试中达到与精确度 ORNN 和 LSTM 相似的结果，即使使用了 3 位量化。