Long Short-Term Memory (LSTM) is a popular approach to boosting the ability
of Recurrent Neural Networks to store longer term temporal information. The
capacity of an LSTM network can be increased by widening and adding layers.
However, usually the former introduces additional parameters, while the latter
increases the runtime. As an alternative we propose the Tensorized LSTM in
which the hidden states are represented by tensors and updated via a
cross-layer convolution. By increasing the tensor size, the network can be
widened efficiently without additional parameters since the parameters are
shared across different locations in the tensor; by delaying the output, the
network can be deepened implicitly with little additional runtime since deep
computations for each timestep are merged into temporal computations of the
sequence. Experiments conducted on five challenging sequence learning tasks
show the potential of the proposed model.

本文介绍了一种新的神经网络模型，即 Tensorized LSTM，它使用张量来表示隐藏状态，通过跨层卷积来更新这些状态。该模型在不增加额外参数的情况下，有效地扩展了网络的容量，并将深度计算合并为时序计算，从而提高了其性能。实验结果表明，该模型在多项挑战性的序列学习任务中表现出很大的潜力。