Weakly-supervised audio-visual violence detection aims to distinguish
snippets containing multimodal violence events with video-level labels. Many
prior works perform audio-visual integration and interaction in an early or
intermediate manner, yet overlooking the modality heterogeneousness over the
weakly-supervised setting. In this paper, we analyze the modality asynchrony
and undifferentiated instances phenomena of the multiple instance learning
(MIL) procedure, and further investigate its negative impact on
weakly-supervised audio-visual learning. To address these issues, we propose a
modality-aware contrastive instance learning with self-distillation (MACIL-SD)
strategy. Specifically, we leverage a lightweight two-stream network to
generate audio and visual bags, in which unimodal background, violent, and
normal instances are clustered into semi-bags in an unsupervised way. Then
audio and visual violent semi-bag representations are assembled as positive
pairs, and violent semi-bags are combined with background and normal instances
in the opposite modality as contrastive negative pairs. Furthermore, a
self-distillation module is applied to transfer unimodal visual knowledge to
the audio-visual model, which alleviates noises and closes the semantic gap
between unimodal and multimodal features. Experiments show that our framework
outperforms previous methods with lower complexity on the large-scale
XD-Violence dataset. Results also demonstrate that our proposed approach can be
used as plug-in modules to enhance other networks. Codes are available at
this https URL

本文提出一种基于弱监督学习的音视频暴力检测方法，通过模态感知对比学习与自我蒸馏策略，有效地提高了音视频一致性，克服了多通道学习中的异质性问题，并在大规模 XD-Violence 数据集上达到更优的检测性能。

基于模态感知的对比实例学习与自监督蒸馏用于弱监督的音视频暴力检测

Modality-Aware Contrastive Instance Learning with Self-Distillation for  Weakly-Supervised Audio-Visual Violence Detection

The journey of reducing noise from distant supervision (DS) generated
training data has been started since the DS was first introduced into the
relation extraction (RE) task. For the past decade, researchers apply the
multi-instance learning (MIL) framework to find the most reliable feature from
a bag of sentences. Although the pattern of MIL bags can greatly reduce DS
noise, it fails to represent many other useful sentence features in the
datasets. In many cases, these sentence features can only be acquired by extra
sentence-level human annotation with heavy costs. Therefore, the performance of
distantly supervised RE models is bounded. In this paper, we go beyond typical
MIL framework and propose a novel contrastive instance learning (CIL)
framework. Specifically, we regard the initial MIL as the relational triple
encoder and constraint positive pairs against negative pairs for each instance.
Experiments demonstrate the effectiveness of our proposed framework, with
significant improvements over the previous methods on NYT10, GDS and KBP.

本文提出了一种新的对比实例学习框架，用于减少远程监督生成的训练数据中的噪声。与传统的多实例学习框架不同，该框架将初步多实例学习作为关系三元组编码器，并针对每个实例的正负对进行限制，实验证明该方法在 NYT10、GDS 和 KBP 上较以前的方法有显著提高。

面向远程监督关系抽取的对比实例学习框架

CIL: Contrastive Instance Learning Framework for Distantly Supervised  Relation Extraction

Self-supervised representation learning has seen remarkable progress in the
last few years. More recently, contrastive instance learning has shown
impressive results compared to its supervised learning counterparts. However,
even with the ever increased interest in contrastive instance learning, it is
still largely unclear why these methods work so well. In this paper, we aim to
unravel some of the mysteries behind their success, which are the good
practices. Through an extensive empirical analysis, we hope to not only provide
insights but also lay out a set of best practices that led to the success of
recent work in self-supervised representation learning.

本文通过对大量实证分析，分析了最近在自我监督表征学习的成功背后背后的最佳实践，发现对比实例学习在无监督学习中表现出色。