Video anomaly detection is a challenging task in the computer vision
community. Most single task-based methods do not consider the independence of
unique spatial and temporal patterns, while two-stream structures lack the
exploration of the correlations. In this paper, we propose spatial-temporal
memories augmented two-stream auto-encoder framework, which learns the
appearance normality and motion normality independently and explores the
correlations via adversarial learning. Specifically, we first design two proxy
tasks to train the two-stream structure to extract appearance and motion
features in isolation. Then, the prototypical features are recorded in the
corresponding spatial and temporal memory pools. Finally, the encoding-decoding
network performs adversarial learning with the discriminator to explore the
correlations between spatial and temporal patterns. Experimental results show
that our framework outperforms the state-of-the-art methods, achieving AUCs of
98.1% and 89.8% on UCSD Ped2 and CUHK Avenue datasets.

本文提出了一种增强空间 - 时间记忆的双流自编码器框架，通过对抗学习，学习外观正常性和运动正常性，并探索空间和时间模式之间的相关性，在 UCSD Ped2 和 CUHK Avenue 数据集上，该框架的 AUC 分别为 98.1％和 89.8％，胜过最先进的方法。