Speech foundation models, trained on vast datasets, have opened unique
opportunities in addressing challenging low-resource speech understanding, such
as child speech. In this work, we explore the capabilities of speech foundation
models on child-adult speaker diarization. We show that exemplary foundation
models can achieve 39.5% and 62.3% relative reductions in Diarization Error
Rate and Speaker Confusion Rate, respectively, compared to previous speaker
diarization methods. In addition, we benchmark and evaluate the speaker
diarization results of the speech foundation models with varying the input
audio window size, speaker demographics, and training data ratio. Our results
highlight promising pathways for understanding and adopting speech foundation
models to facilitate child speech understanding.

我们的研究工作探索了言语基础模型在儿童成人说话人分离中的应用能力，并发现出色的基础模型相对于以前的方法在分离错误率和说话人混淆率方面可以实现 39.5% 和 62.3% 的相对降低。此外，我们还评估了输入音频窗口大小、说话人特点和训练数据比例对言语基础模型的分离结果的影响，结果显示有希望通过采用言语基础模型来促进对儿童说话的理解。

探索儿童 - 成人交互中的演讲基础模型用于说话者分离

Exploring Speech Foundation Models for Speaker Diarization in  Child-Adult Dyadic Interactions

Majority of speech signals across different scenarios are never available
with well-defined audio segments containing only a single speaker. A typical
conversation between two speakers consists of segments where their voices
overlap, interrupt each other or halt their speech in between multiple
sentences. Recent advancements in diarization technology leverage neural
network-based approaches to improvise multiple subsystems of speaker
diarization system comprising of extracting segment-wise embedding features and
detecting changes in the speaker during conversation. However, to identify
speaker through clustering, models depend on methodologies like PLDA to
generate similarity measure between two extracted segments from a given
conversational audio. Since these algorithms ignore the temporal structure of
conversations, they tend to achieve a higher Diarization Error Rate (DER), thus
leading to misdetections both in terms of speaker and change identification.
Therefore, to compare similarity of two speech segments both independently and
sequentially, we propose a Bi-directional Long Short-term Memory network for
estimating the elements present in the similarity matrix. Once the similarity
matrix is generated, Agglomerative Hierarchical Clustering (AHC) is applied to
further identify speaker segments based on thresholding. To evaluate the
performance, Diarization Error Rate (DER%) metric is used. The proposed model
achieves a low DER of 34.80% on a test set of audio samples derived from ICSI
Meeting Corpus as compared to traditional PLDA based similarity measurement
mechanism which achieved a DER of 39.90%.

本文提出了一种基于双向 LSTM 网络的语音分割与边界检测方法，用于解决多个人同时说话时 PLDA 算法无法考虑语音结构，容易出错的问题，实验表明该方法在 ICSI 会议语音数据集上的 Diarization Error Rate 可降低至 34.80%。

基于 Bi-LSTM 评分和凝聚层次聚类 (AHC) 的说话人分离相似度测量

Bi-LSTM Scoring Based Similarity Measurement with Agglomerative Hierarchical Clustering (AHC) for Speaker Diarization

In this paper, we propose a fully supervised speaker diarization approach,
named unbounded interleaved-state recurrent neural networks (UIS-RNN). Given
extracted speaker-discriminative embeddings (a.k.a. d-vectors) from input
utterances, each individual speaker is modeled by a parameter-sharing RNN,
while the RNN states for different speakers interleave in the time domain. This
RNN is naturally integrated with a distance-dependent Chinese restaurant
process (ddCRP) to accommodate an unknown number of speakers. Our system is
fully supervised and is able to learn from examples where time-stamped speaker
labels are annotated. We achieved a 7.6% diarization error rate on NIST SRE
2000 CALLHOME, which is better than the state-of-the-art method using spectral
clustering. Moreover, our method decodes in an online fashion while most
state-of-the-art systems rely on offline clustering.

本文提出了一种全监督的说话人分离方法，称为无限交错状态循环神经网络（UIS-RNN），通过 RNN 建模不同说话人，运用了 ddCRP 解决未知说话人数量问题，并实现在线分离。在 NIST SRE 2000 CALLHOME 上，其检测率为 7.6%，优于现有最先进的基于谱聚类的分离方法。