Video prediction is a pixel-level task that generates future frames by
employing the historical frames. There often exist continuous complex motions,
such as object overlapping and scene occlusion in video, which poses great
challenges to this task. Previous works either fail to well capture the
long-term temporal dynamics or do not handle the occlusion masks. To address
these issues, we develop the fully convolutional Fast Fourier Inception
Networks for video prediction, termed \textit{FFINet}, which includes two
primary components, \ie, the occlusion inpainter and the spatiotemporal
translator. The former adopts the fast Fourier convolutions to enlarge the
receptive field, such that the missing areas (occlusion) with complex geometric
structures are filled by the inpainter. The latter employs the stacked Fourier
transform inception module to learn the temporal evolution by group
convolutions and the spatial movement by channel-wise Fourier convolutions,
which captures both the local and the global spatiotemporal features. This
encourages generating more realistic and high-quality future frames. To
optimize the model, the recovery loss is imposed to the objective, \ie,
minimizing the mean square error between the ground-truth frame and the
recovery frame. Both quantitative and qualitative experimental results on five
benchmarks, including Moving MNIST, TaxiBJ, Human3.6M, Caltech Pedestrian, and
KTH, have demonstrated the superiority of the proposed approach. Our code is
available at GitHub.

为了解决视频帧预测任务中的连续复杂运动和场景遮挡问题，我们开发了基于全卷积快速傅里叶 Inception 网络（FFINet）的视频预测方法，该方法包括遮挡修复器和时空转换器等两个组件，并将恢复损失纳入目标函数进行模型优化来生成更逼真、高质量的未来帧。