Video Anomaly Detection (VAD), aiming to identify abnormalities within a
specific context and timeframe, is crucial for intelligent Video Surveillance
Systems. While recent deep learning-based VAD models have shown promising
results by generating high-resolution frames, they often lack competence in
preserving detailed spatial and temporal coherence in video frames. To tackle
this issue, we propose a self-supervised learning approach for VAD through an
inter-patch relationship prediction task. Specifically, we introduce a
two-branch vision transformer network designed to capture deep visual features
of video frames, addressing spatial and temporal dimensions responsible for
modeling appearance and motion patterns, respectively. The inter-patch
relationship in each dimension is decoupled into inter-patch similarity and the
order information of each patch. To mitigate memory consumption, we convert the
order information prediction task into a multi-label learning problem, and the
inter-patch similarity prediction task into a distance matrix regression
problem. Comprehensive experiments demonstrate the effectiveness of our method,
surpassing pixel-generation-based methods by a significant margin across three
public benchmarks. Additionally, our approach outperforms other self-supervised
learning-based methods.

通过自监督学习和视觉变换网络，提出一种用于视频异常检测的方法，能够有效地保留视频帧的空间和时间上的一致性，并且在三个公共基准测试中表现优于像素生成方法和其他自监督学习方法。