Speaker segmentation consists in partitioning a conversation between one or
more speakers into speaker turns. Usually addressed as the late combination of
three sub-tasks (voice activity detection, speaker change detection, and
overlapped speech detection), we propose to train an end-to-end segmentation
model that does it directly. Inspired by the original end-to-end neural speaker
diarization approach (EEND), the task is modeled as a multi-label
classification problem using permutation-invariant training. The main
difference is that our model operates on short audio chunks (5 seconds) but at
a much higher temporal resolution (every 16ms). Experiments on multiple speaker
diarization datasets conclude that our model can be used with great success on
both voice activity detection and overlapped speech detection. Our proposed
model can also be used as a post-processing step, to detect and correctly
assign overlapped speech regions. Relative diarization error rate improvement
over the best considered baseline (VBx) reaches 17% on AMI, 13% on DIHARD 3,
and 13% on VoxConverse.

提出了一种基于端到端模型的说话人分割方法，直接进行说话人划分，通过多标签分类解决此任务，同时可用于语音活动检测和重叠语音检测，且在多个数据集上都有显著的表现提升。