Detecting abnormal activities in real-world surveillance videos is an
important yet challenging task as the prior knowledge about video anomalies is
usually limited or unavailable. Despite that many approaches have been
developed to resolve this problem, few of them can capture the normal
spatio-temporal patterns effectively and efficiently. Moreover, existing works
seldom explicitly consider the local consistency at frame level and global
coherence of temporal dynamics in video sequences. To this end, we propose
Convolutional Transformer based Dual Discriminator Generative Adversarial
Networks (CT-D2GAN) to perform unsupervised video anomaly detection.
Specifically, we first present a convolutional transformer to perform future
frame prediction. It contains three key components, i.e., a convolutional
encoder to capture the spatial information of the input video clips, a temporal
self-attention module to encode the temporal dynamics, and a convolutional
decoder to integrate spatio-temporal features and predict the future frame.
Next, a dual discriminator based adversarial training procedure, which jointly
considers an image discriminator that can maintain the local consistency at
frame-level and a video discriminator that can enforce the global coherence of
temporal dynamics, is employed to enhance the future frame prediction. Finally,
the prediction error is used to identify abnormal video frames. Thoroughly
empirical studies on three public video anomaly detection datasets, i.e., UCSD
Ped2, CUHK Avenue, and Shanghai Tech Campus, demonstrate the effectiveness of
the proposed adversarial spatio-temporal modeling framework.

本文介绍了一种基于卷积变换器、双鉴别器生成对抗网络（CT-D2GAN）的非监督式视频异常检测方法，该方法在未来帧预测和对视频的局部和全局建模方面取得了显著的效果。

基于卷积变换器和双重鉴别器的生成对抗网络，用于视频异常检测

Convolutional Transformer based Dual Discriminator Generative  Adversarial Networks for Video Anomaly Detection

We propose in this paper a novel approach to tackle the problem of mode
collapse encountered in generative adversarial network (GAN). Our idea is
intuitive but proven to be very effective, especially in addressing some key
limitations of GAN. In essence, it combines the Kullback-Leibler (KL) and
reverse KL divergences into a unified objective function, thus it exploits the
complementary statistical properties from these divergences to effectively
diversify the estimated density in capturing multi-modes. We term our method
dual discriminator generative adversarial nets (D2GAN) which, unlike GAN, has
two discriminators; and together with a generator, it also has the analogy of a
minimax game, wherein a discriminator rewards high scores for samples from data
distribution whilst another discriminator, conversely, favoring data from the
generator, and the generator produces data to fool both two discriminators. We
develop theoretical analysis to show that, given the maximal discriminators,
optimizing the generator of D2GAN reduces to minimizing both KL and reverse KL
divergences between data distribution and the distribution induced from the
data generated by the generator, hence effectively avoiding the mode collapsing
problem. We conduct extensive experiments on synthetic and real-world
large-scale datasets (MNIST, CIFAR-10, STL-10, ImageNet), where we have made
our best effort to compare our D2GAN with the latest state-of-the-art GAN's
variants in comprehensive qualitative and quantitative evaluations. The
experimental results demonstrate the competitive and superior performance of
our approach in generating good quality and diverse samples over baselines, and
the capability of our method to scale up to ImageNet database.

本文提出了一种名为双判别器生成对抗网络（D2GAN）的生成对抗网络算法，该算法使用 KL 散度和反 KL 散度，避免了多峰性分布的崩塌问题，并在广泛的实验中证明了与最新 GAN 算法相比的竞争和卓越性能。