This paper introduces Grid Long Short-Term Memory, a network of LSTM cells arranged in a multidimensional grid that can be applied to vectors, sequences or higher dimensional data such as images. The network differs from existing deep LSTM architectures in that the cells are connected between network layers as well as along the spatiotemporal dimensions of the data. It therefore provides a unified way of using LSTM for both deep and sequential computation. We apply the model to algorithmic tasks such as integer addition and determining the parity of random binary vectors. It is able to solve these problems for 15-digit integers and 250-bit vectors respectively. We then give results for three empirical tasks. We find that 2D Grid LSTM achieves 1.47 bits per character on the Wikipedia character prediction benchmark, which is state-of-the-art among neural approaches. We also observe that a two-dimensional translation model based on Grid LSTM outperforms a phrase-based reference system on a Chinese-to-English translation task, and that 3D Grid LSTM yields a near state-of-the-art error rate of 0.32% on MNIST.

本文介绍了Grid Long Short-Term Memory，这是一种由LSTM单元组成的多维网格网络，可用于向量、序列或图像等更高维的数据。该网络在现有深度LSTM架构中的不同之处在于单元不仅在网络层之间连接，也连接在数据的时空维度上，提供了一种统一的使用LSTM进行深层次和序列计算的方法。我们将模型应用于15位整数加法和序列记忆等算法任务，发现其能够显著优于标准LSTM。我们还给出了两个实证任务的结果。2D Grid LSTM在维基百科字符预测基准测试中达到了1.47位/字符，这是神经方法中的最佳结果。此外，我们使用Grid LSTM定义了一种新的二维翻译模型Reencoder，并表明其在中英翻译任务上优于基于短语的参考系统。

网格长短时记忆