We introduce a new zero resource code-switched speech benchmark designed to
directly assess the code-switching capabilities of self-supervised speech
encoders. We showcase a baseline system of language modeling on discrete units
to demonstrate how the code-switching abilities of speech encoders can be
assessed in a zero-resource manner. Our experiments encompass a variety of
well-known speech encoders, including Wav2vec 2.0, HuBERT, XLSR, etc. We
examine the impact of pre-training languages and model size on benchmark
performance. Notably, though our results demonstrate that speech encoders with
multilingual pre-training, exemplified by XLSR, outperform monolingual variants
(Wav2vec 2.0, HuBERT) in code-switching scenarios, there is still substantial
room for improvement in their code-switching linguistic abilities.

我们介绍了一个新的零资源代码切换语音基准，旨在直接评估自我监督语音编码器的代码切换能力。我们展示了一个以离散单元上的语言建模为基线系统，以演示如何以零资源的方式评估语音编码器的代码切换能力。我们的实验包括多种知名的语音编码器，包括 Wav2vec 2.0、HuBERT、XLSR 等。我们研究了预训练语言和模型大小对基准性能的影响。值得注意的是，尽管我们的结果表明，在代码切换场景中，具有多语言预训练的语音编码器（如 XLSR）优于单语变体（Wav2vec 2.0、HuBERT），但它们的代码切换语言能力仍有很大的改进空间。