We present a novel end-to-end partially supervised deep learning approach for
video anomaly detection and localization using only normal samples. The insight
that motivates this study is that the normal samples can be associated with at
least one Gaussian component of a Gaussian Mixture Model (GMM), while anomalies
either do not belong to any Gaussian component. The method is based on Gaussian
Mixture Variational Autoencoder, which can learn feature representations of the
normal samples as a Gaussian Mixture Model trained using deep learning. A Fully
Convolutional Network (FCN) that does not contain a fully-connected layer is
employed for the encoder-decoder structure to preserve relative spatial
coordinates between the input image and the output feature map. Based on the
joint probabilities of each of the Gaussian mixture components, we introduce a
sample energy based method to score the anomaly of image test patches. A
two-stream network framework is employed to combine the appearance and motion
anomalies, using RGB frames for the former and dynamic flow images, for the
latter. We test our approach on two popular benchmarks (UCSD Dataset and Avenue
Dataset). The experimental results verify the superiority of our method
compared to the state of the arts.

提出了一种部分有监督的深度学习方法，用于视频异常检测和定位，其中使用的仅为正常样本，并且该方法基于使用深度学习训练的高斯混合模型，可以学习正常样本的特征表示。通过使用 RGB 帧和动态流图像，同时使用出现和运动异常构建了一个双流网络框架，并基于高斯混合分量的联合概率引入了一种基于样本能量的方法来评分图像测试块的异常。在 UCSD 数据集和 Avenue 数据集上进行了测试实验，结果表明，与现有技术相比，该方法具有卓越的性能。