Video prediction aims to predict future frames from a video's previous
content. Existing methods mainly process video data where the time dimension
mingles with the space and channel dimensions from three distinct angles: as a
sequence of individual frames, as a 3D volume in spatiotemporal coordinates, or
as a stacked image where frames are treated as separate channels. Most of them
generally focus on one of these perspectives and may fail to fully exploit the
relationships across different dimensions. To address this issue, this paper
introduces a convolutional mixer for video prediction, termed ViP-Mixer, to
model the spatiotemporal evolution in the latent space of an autoencoder. The
ViP-Mixers are stacked sequentially and interleave feature mixing at three
levels: frames, channels, and locations. Extensive experiments demonstrate that
our proposed method achieves new state-of-the-art prediction performance on
three benchmark video datasets covering both synthetic and real-world
scenarios.

本文提出了一种名为 ViP-Mixer 的卷积混合器，用于视频预测，模拟了自编码器潜在空间中的时空演变，并通过在帧、通道和位置三个层次交错特征混合，取得了在包括合成和真实场景的三个基准视频数据集上的新的预测性能的最新成果。