Deep neural networks (DNNs) have achieved outstanding performance in a wide
range of applications, e.g., image classification, natural language processing,
etc. Despite the good performance, the huge number of parameters in DNNs brings
challenges to efficient training of DNNs and also their deployment in low-end
devices with limited computing resources. In this paper, we explore the
correlations in the weight matrices, and approximate the weight matrices with
the low-rank block-term tensors. We name the new corresponding structure as
block-term tensor layers (BT-layers), which can be easily adapted to neural
network models, such as CNNs and RNNs. In particular, the inputs and the
outputs in BT-layers are reshaped into low-dimensional high-order tensors with
a similar or improved representation power. Sufficient experiments have
demonstrated that BT-layers in CNNs and RNNs can achieve a very large
compression ratio on the number of parameters while preserving or improving the
representation power of the original DNNs.

本文提出了一种名为 BT-layers 的新的神经网络结构，使用低秩分块张量来近似权重矩阵，能够大幅缩小 DNN 的参数数量，同时保持或提高其表征能力。