Self-supervised (SSL) models have shown great performance in various
downstream tasks. However, they are typically developed for limited languages,
and may encounter new languages in real-world. Developing a SSL model for each
new language is costly. Thus, it is vital to figure out how to efficiently
adapt existed SSL models to a new language without impairing its original
abilities. We propose adaptation methods which integrate LoRA to existed SSL
models to extend new language. We also develop preservation strategies which
include data combination and re-clustering to retain abilities on existed
languages. Applied to mHuBERT, we investigate their effectiveness on speech
re-synthesis task. Experiments show that our adaptation methods enable mHuBERT
to be applied to a new language (Mandarin) with MOS value increased about 1.6
and the relative value of WER reduced up to 61.72%. Also, our preservation
strategies ensure that the performance on both existed and new languages
remains intact.

我们提出了通过将 LoRA 整合到已有的自我监督模型中，实现新语言的自适应，并通过数据结合和重新聚类等保护策略来保持对已有和新语言的性能保持完整，实验证明该方法使 mHuBERT 应用于新语言（普通话）的 MOS 值增加 1.6，WER 相对值减少了 61.72%。