Recurrent Neural Networks (RNNs) have been widely used in sequence analysis
and modeling. However, when processing high-dimensional data, RNNs typically
require very large model sizes, thereby bringing a series of deployment
challenges. Although the state-of-the-art tensor decomposition approaches can
provide good model compression performance, these existing methods are still
suffering some inherent limitations, such as restricted representation
capability and insufficient model complexity reduction. To overcome these
limitations, in this paper we propose to develop compact RNN models using
Hierarchical Tucker (HT) decomposition. HT decomposition brings strong
hierarchical structure to the decomposed RNN models, which is very useful and
important for enhancing the representation capability. Meanwhile, HT
decomposition provides higher storage and computational cost reduction than the
existing tensor decomposition approaches for RNN compression. Our experimental
results show that, compared with the state-of-the-art compressed RNN models,
such as TT-LSTM, TR-LSTM and BT-LSTM, our proposed HT-based LSTM (HT-LSTM),
consistently achieves simultaneous and significant increases in both
compression ratio and test accuracy on different datasets.

本研究提出使用分层 Tucker 分解压缩循环神经网络，提高了其表示能力且相比于 TT-LSTM、TR-LSTM、BT-LSTM 等现有压缩模型，在不同数据集上同时实现了压缩比和测试准确度的显著提升。