Video anomaly detection (VAD) with weak supervision has achieved remarkable
performance in utilizing video-level labels to discriminate whether a video
frame is normal or abnormal. However, current approaches are inherently limited
to a closed-set setting and may struggle in open-world applications where there
can be anomaly categories in the test data unseen during training. A few recent
studies attempt to tackle a more realistic setting, open-set VAD, which aims to
detect unseen anomalies given seen anomalies and normal videos. However, such a
setting focuses on predicting frame anomaly scores, having no ability to
recognize the specific categories of anomalies, despite the fact that this
ability is essential for building more informed video surveillance systems.
This paper takes a step further and explores open-vocabulary video anomaly
detection (OVVAD), in which we aim to leverage pre-trained large models to
detect and categorize seen and unseen anomalies. To this end, we propose a
model that decouples OVVAD into two mutually complementary tasks --
class-agnostic detection and class-specific classification -- and jointly
optimizes both tasks. Particularly, we devise a semantic knowledge injection
module to introduce semantic knowledge from large language models for the
detection task, and design a novel anomaly synthesis module to generate pseudo
unseen anomaly videos with the help of large vision generation models for the
classification task. These semantic knowledge and synthesis anomalies
substantially extend our model's capability in detecting and categorizing a
variety of seen and unseen anomalies. Extensive experiments on three
widely-used benchmarks demonstrate our model achieves state-of-the-art
performance on OVVAD task.

这篇论文提出了一种能够利用预训练的大型模型来检测和分类已知和未知异常的开放词汇视频异常检测方法，并通过引入语义知识和合成异常来提高模型的性能。