Overlapped speech detection (OSD) is critical for speech applications in
scenario of multi-party conversion. Despite numerous research efforts and
progresses, comparing with speech activity detection (VAD), OSD remains an open
challenge and its overall performance is far from satisfactory. The majority of
prior research typically formulates the OSD problem as a standard
classification problem, to identify speech with binary (OSD) or three-class
label (joint VAD and OSD) at frame level. In contrast to the mainstream, this
study investigates the joint VAD and OSD task from a new perspective. In
particular, we propose to extend traditional classification network with
multi-exit architecture. Such an architecture empowers our system with unique
capability to identify class using either low-level features from early exits
or high-level features from last exit. In addition, two training schemes,
knowledge distillation and dense connection, are adopted to further boost our
system performance. Experimental results on benchmark datasets (AMI and
DIHARD-III) validated the effectiveness and generality of our proposed system.
Our ablations further reveal the complementary contribution of proposed
schemes. With $F_1$ score of 0.792 on AMI and 0.625 on DIHARD-III, our proposed
system outperforms several top performing models on these datasets, but also
surpasses the current state-of-the-art by large margins across both datasets.
Besides the performance benefit, our proposed system offers another appealing
potential for quality-complexity trade-offs, which is highly preferred for
efficient OSD deployment.

本文提出了一种采用多出口架构、知识蒸馏和密集连接等方法的音频多方对话中的叠声检测模型，该模型在 AMI 和 DIHARD-III 基准数据集上的实验结果验证了其有效性和泛化性，并在质量 - 复杂度平衡方面提供了可行的选择。

多出口结构联合语音活动和重叠检测

Joint Speech Activity and Overlap Detection with Multi-Exit Architecture

Contrastive loss has significantly improved performance in supervised
classification tasks by using a multi-viewed framework that leverages
augmentation and label information. The augmentation enables contrast with
another view of a single image but enlarges training time and memory usage. To
exploit the strength of multi-views while avoiding the high computation cost,
we introduce a multi-exit architecture that outputs multiple features of a
single image in a single-viewed framework. To this end, we propose
Self-Contrastive (SelfCon) learning, which self-contrasts within multiple
outputs from the different levels of a single network. The multi-exit
architecture efficiently replaces multi-augmented images and leverages various
information from different layers of a network. We demonstrate that SelfCon
learning improves the classification performance of the encoder network, and
empirically analyze its advantages in terms of the single-view and the
sub-network. Furthermore, we provide theoretical evidence of the performance
increase based on the mutual information bound. For ImageNet classification on
ResNet-50, SelfCon improves accuracy by +0.6% with 59% memory and 48% time of
Supervised Contrastive learning, and a simple ensemble of multi-exit outputs
boosts performance up to +1.5%. Our code is available at
this https URL

本研究介绍了自同对比学习 (SelfContrastive) 和多出口模型 (multi-exit architecture)，以达到在 ResNet-50 上进行 ImageNet 分类的目的。SelfCon 通过多路单一网络计算出多个图像特征来替代传统的多视图方法，其精度相较于传统方法提升了 0.6%，同时只需用传统方法的 59% 的内存和 48% 的训练时间。