In a code-switched (CS) scenario, the use of spoken language diarization (LD)
as a pre-possessing system is essential. Further, the use of implicit
frameworks is preferable over the explicit framework, as it can be easily
adapted to deal with low/zero resource languages. Inspired by speaker
diarization (SD) literature, three frameworks based on (1) fixed segmentation,
(2) change point-based segmentation and (3) E2E are proposed to perform LD. The
initial exploration with synthetic TTSF-LD dataset shows, using x-vector as
implicit language representation with appropriate analysis window length ($N$)
can able to achieve at per performance with explicit LD. The best implicit LD
performance of $6.38$ in terms of Jaccard error rate (JER) is achieved by using
the E2E framework. However, considering the E2E framework the performance of
implicit LD degrades to $60.4$ while using with practical Microsoft CS (MSCS)
dataset. The difference in performance is mostly due to the distributional
difference between the monolingual segment duration of secondary language in
the MSCS and TTSF-LD datasets. Moreover, to avoid segment smoothing, the
smaller duration of the monolingual segment suggests the use of a small value
of $N$. At the same time with small $N$, the x-vector representation is unable
to capture the required language discrimination due to the acoustic similarity,
as the same speaker is speaking both languages. Therefore, to resolve the issue
a self-supervised implicit language representation is proposed in this study.
In comparison with the x-vector representation, the proposed representation
provides a relative improvement of $63.9\%$ and achieved a JER of $21.8$ using
the E2E framework.

在一个混合语言环境中，使用隐性框架的语音语言编组（LD）作为预处理系统是必要的。通过基于固定分割、基于变化点分割和 E2E 的三种框架，提出了三种实现 LD 的方法。而当使用微软 CS（MSCS）数据集时，使用 E2E 框架的隐性 LD 的性能下降至 60.4%，主要是由于 MSCS 和 TTSF-LD 数据集中辅助语言的单语片段持续时间的分布差异。因此，为解决这个问题，本研究提出了一种自监督的隐性语言表示方法，相对于 x-vector 表示，该方法实现了约 63.9% 的改进，并在 E2E 框架下取得了 21.8 的 JER。

自我监督的口语语言表示在语音语言分离中的应用

Implicit Self-supervised Language Representation for Spoken Language  Diarization

Spoken language diarization (LD) and related tasks are mostly explored using
the phonotactic approach. Phonotactic approaches mostly use explicit way of
language modeling, hence requiring intermediate phoneme modeling and
transcribed data. Alternatively, the ability of deep learning approaches to
model temporal dynamics may help for the implicit modeling of language
information through deep embedding vectors. Hence this work initially explores
the available speaker diarization frameworks that capture speaker information
implicitly to perform LD tasks. The performance of the LD system on synthetic
code-switch data using the end-to-end x-vector approach is 6.78% and 7.06%, and
for practical data is 22.50% and 60.38%, in terms of diarization error rate and
Jaccard error rate (JER), respectively. The performance degradation is due to
the data imbalance and resolved to some extent by using pre-trained wave2vec
embeddings that provide a relative improvement of 30.74% in terms of JER.

本研究探讨了利用深度学习的端到端 x - 向量方法进行语音流言分离的性能表现，并发现预训练的 wave2vec 嵌入可以提供近 30% 的错误率改进。