Learning from spatio-temporal data has numerous applications such as
human-behavior analysis, object tracking, video compression, and physics
simulation.However, existing methods still perform poorly on challenging video
tasks such as long-term forecasting. This is because these kinds of challenging
tasks require learning long-term spatio-temporal correlations in the video
sequence. In this paper, we propose a higher-order convolutional LSTM model
that can efficiently learn these correlations, along with a succinct
representations of the history. This is accomplished through a novel tensor
train module that performs prediction by combining convolutional features
across time. To make this feasible in terms of computation and memory
requirements, we propose a novel convolutional tensor-train decomposition of
the higher-order model. This decomposition reduces the model complexity by
jointly approximating a sequence of convolutional kernels asa low-rank
tensor-train factorization. As a result, our model outperforms existing
approaches, but uses only a fraction of parameters, including the baseline
models.Our results achieve state-of-the-art performance in a wide range of
applications and datasets, including the multi-steps video prediction on the
Moving-MNIST-2and KTH action datasets as well as early activity recognition on
the Something-Something V2 dataset.

本文提出了一种高阶卷积 LSTM 模型和一种新的张量列车模块，可以有效地学习视频序列中的长期时空相关性，并通过结合时间上的卷积特征进行预测，通过张量列车分解降低了模型复杂度，取得了广泛应用和数据集上的最佳表现。