Weakly supervised multimodal violence detection aims to learn a violence
detection model by leveraging multiple modalities such as RGB, optical flow,
and audio, while only video-level annotations are available. In the pursuit of
effective multimodal violence detection (MVD), information redundancy, modality
imbalance, and modality asynchrony are identified as three key challenges. In
this work, we propose a new weakly supervised MVD method that explicitly
addresses these challenges. Specifically, we introduce a multi-scale bottleneck
transformer (MSBT) based fusion module that employs a reduced number of
bottleneck tokens to gradually condense information and fuse each pair of
modalities and utilizes a bottleneck token-based weighting scheme to highlight
more important fused features. Furthermore, we propose a temporal consistency
contrast loss to semantically align pairwise fused features. Experiments on the
largest-scale XD-Violence dataset demonstrate that the proposed method achieves
state-of-the-art performance. Code is available at
this https URL

通过利用多种多样性的模态，如 RGB，光流和音频，以及仅具备视频级别注释的多模态暴力检测模型的学习，我们提出了一种新的弱监督多模态暴力检测方法，明确解决了信息冗余，模态不平衡和模态异步性等三个关键挑战，进一步提出了一种时间一致性对比损失来语义对齐成对融合特征。在最大规模的 XD-Violence 数据集上的实验表明，提出的方法实现了最先进的性能。