Our work addresses long-term motion context issues for predicting future
frames. To predict the future precisely, it is required to capture which
long-term motion context (e.g., walking or running) the input motion (e.g., leg
movement) belongs to. The bottlenecks arising when dealing with the long-term
motion context are: (i) how to predict the long-term motion context naturally
matching input sequences with limited dynamics, (ii) how to predict the
long-term motion context with high-dimensionality (e.g., complex motion). To
address the issues, we propose novel motion context-aware video prediction. To
solve the bottleneck (i), we introduce a long-term motion context memory
(LMC-Memory) with memory alignment learning. The proposed memory alignment
learning enables to store long-term motion contexts into the memory and to
match them with sequences including limited dynamics. As a result, the
long-term context can be recalled from the limited input sequence. In addition,
to resolve the bottleneck (ii), we propose memory query decomposition to store
local motion context (i.e., low-dimensional dynamics) and recall the suitable
local context for each local part of the input individually. It enables to
boost the alignment effects of the memory. Experimental results show that the
proposed method outperforms other sophisticated RNN-based methods, especially
in long-term condition. Further, we validate the effectiveness of the proposed
network designs by conducting ablation studies and memory feature analysis. The
source code of this work is available.

该研究提出了一种基于长期记忆机制的运动上下文感知视频预测方法，通过引入长期运动上下文记忆（LMC-Memory）和记忆对查询分解，解决了在低维动态输入情况下预测高维、复杂运动上下文的难题。实验证明，该方法在长期情况下优于其他 RNN-based 方法。

视频预测：通过记忆对齐学习长期运动上下文

Video Prediction Recalling Long-term Motion Context via Memory Alignment  Learning

Video based person re-identification plays a central role in realistic
security and video surveillance. In this paper we propose a novel Accumulative
Motion Context (AMOC) network for addressing this important problem, which
effectively exploits the long-range motion context for robustly identifying the
same person under challenging conditions. Given a video sequence of the same or
different persons, the proposed AMOC network jointly learns appearance
representation and motion context from a collection of adjacent frames using a
two-stream convolutional architecture. Then AMOC accumulates clues from motion
context by recurrent aggregation, allowing effective information flow among
adjacent frames and capturing dynamic gist of the persons. The architecture of
AMOC is end-to-end trainable and thus motion context can be adapted to
complement appearance clues under unfavorable conditions (e.g. occlusions).
Extensive experiments are conduced on three public benchmark datasets, i.e.,
the iLIDS-VID, PRID-2011 and MARS datasets, to investigate the performance of
AMOC. The experimental results demonstrate that the proposed AMOC network
outperforms state-of-the-arts for video-based re-identification significantly
and confirm the advantage of exploiting long-range motion context for video
based person re-identification, validating our motivation evidently.

本文提出了一种新颖的累积运动上下文（AMOC）网络，它可以有效地利用远距离的运动上下文，在具有挑战性的条件下鲁棒地识别出同一人，并且其架构是端到端可训练的，并且该方法在三个公共数据集上优于现有的最佳水平。