Speaker embedding extractors significantly influence the performance of
clustering-based speaker diarisation systems. Conventionally, only one
embedding is extracted from each speech segment. However, because of the
sliding window approach, a segment easily includes two or more speakers owing
to speaker change points. This study proposes a novel embedding extractor
architecture, referred to as a high-resolution embedding extractor (HEE), which
extracts multiple high-resolution embeddings from each speech segment. Hee
consists of a feature-map extractor and an enhancer, where the enhancer with
the self-attention mechanism is the key to success. The enhancer of HEE
replaces the aggregation process; instead of a global pooling layer, the
enhancer combines relative information to each frame via attention leveraging
the global context. Extracted dense frame-level embeddings can each represent a
speaker. Thus, multiple speakers can be represented by different frame-level
features in each segment. We also propose an artificially generating mixture
data training framework to train the proposed HEE. Through experiments on five
evaluation sets, including four public datasets, the proposed HEE demonstrates
at least 10% improvement on each evaluation set, except for one dataset, which
we analyse that rapid speaker changes less exist.

本文提出了一种新的嵌入提取器体系结构，称为高分辨率嵌入提取器（HEE），它从每个语音片段中提取多个高分辨率嵌入，每个密集的帧级嵌入可以代表一个说话者，多个说话者可以由每个段的不同帧级特征表示。通过在五个评估集上的实验，表明了所提出的 HEE 的至少 10% 的改进。

高分辨率说话人分离嵌入提取器

High-resolution embedding extractor for speaker diarisation

Speaker embedding extractors (EEs), which map input audio to a speaker
discriminant latent space, are of paramount importance in speaker diarisation.
However, there are several challenges when adopting EEs for diarisation, from
which we tackle two key problems. First, the evaluation is not straightforward
because the features required for better performance differ between speaker
verification and diarisation. We show that better performance on widely adopted
speaker verification evaluation protocols does not lead to better diarisation
performance. Second, embedding extractors have not seen utterances in which
multiple speakers exist. These inputs are inevitably present in speaker
diarisation because of overlapped speech and speaker changes; they degrade the
performance. To mitigate the first problem, we generate speaker verification
evaluation protocols that mimic the diarisation scenario better. We propose two
data augmentation techniques to alleviate the second problem, making embedding
extractors aware of overlapped speech or speaker change input. One technique
generates overlapped speech segments, and the other generates segments where
two speakers utter sequentially. Extensive experimental results using three
state-of-the-art speaker embedding extractors demonstrate that both proposed
approaches are effective.

通过生成更贴近对话场景的说话人验证评估协议和两种数据增强技术来解决说话人嵌入提取器在多说话人存在时的性能下降问题，实验证明这两种方法都是有效的。

寻找适用于说话人分离的强嵌入提取器

In search of strong embedding extractors for speaker diarisation

The goal of this paper is speaker diarisation of videos collected 'in the
wild'. We make three key contributions. First, we propose an automatic
audio-visual diarisation method for YouTube videos. Our method consists of
active speaker detection using audio-visual methods and speaker verification
using self-enrolled speaker models. Second, we integrate our method into a
semi-automatic dataset creation pipeline which significantly reduces the number
of hours required to annotate videos with diarisation labels. Finally, we use
this pipeline to create a large-scale diarisation dataset called VoxConverse,
collected from 'in the wild' videos, which we will release publicly to the
research community. Our dataset consists of overlapping speech, a large and
diverse speaker pool, and challenging background conditions.

本研究旨在使用自动化音频视觉分离方法对 YouTube 视频进行说话者分割。研究将自己设计的说话者模型运用于半自动数据集创建流程中，从而显著减少注释视频所需的时间。通过这种方式，研究人员成功地发布了一个名为 VoxConverse 的大规模分离数据集。