Video anomaly detection (VAD) is a significant computer vision problem.
Existing deep neural network (DNN) based VAD methods mostly follow the route of
frame reconstruction or frame prediction. However, the lack of mining and
learning of higher-level visual features and temporal context relationships in
videos limits the further performance of these two approaches. Inspired by
video codec theory, we introduce a brand-new VAD paradigm to break through
these limitations: First, we propose a new task of video event restoration
based on keyframes. Encouraging DNN to infer missing multiple frames based on
video keyframes so as to restore a video event, which can more effectively
motivate DNN to mine and learn potential higher-level visual features and
comprehensive temporal context relationships in the video. To this end, we
propose a novel U-shaped Swin Transformer Network with Dual Skip Connections
(USTN-DSC) for video event restoration, where a cross-attention and a temporal
upsampling residual skip connection are introduced to further assist in
restoring complex static and dynamic motion object features in the video. In
addition, we propose a simple and effective adjacent frame difference loss to
constrain the motion consistency of the video sequence. Extensive experiments
on benchmarks demonstrate that USTN-DSC outperforms most existing methods,
validating the effectiveness of our method.

该研究通过引入新的视频事件修复任务，提出了一种新的视频异常检测范式，其基于关键帧，使用具有双跳连接的 U 形 Swin Transformer 网络进行视频事件修复，并使用相邻帧差异损失约束视频序列的运动一致性，实验结果表明该方法比大多数现有方法表现更优秀。