Representations from pre-trained speech foundation models (SFMs) have shown
impressive performance in many downstream tasks. However, the potential
benefits of incorporating pre-trained SFM representations into speaker voice
similarity assessment have not been thoroughly investigated. In this paper, we
propose SVSNet+, a model that integrates pre-trained SFM representations to
improve performance in assessing speaker voice similarity. Experimental results
on the Voice Conversion Challenge 2018 and 2020 datasets show that SVSNet+
incorporating WavLM representations shows significant improvements compared to
baseline models. In addition, while fine-tuning WavLM with a small dataset of
the downstream task does not improve performance, using the same dataset to
learn a weighted-sum representation of WavLM can substantially improve
performance. Furthermore, when WavLM is replaced by other SFMs, SVSNet+ still
outperforms the baseline models and exhibits strong generalization ability.

本研究提出了 SVSNet + 模型，该模型整合了预训练的 SFMs 模型来改善对说话者声音相似性的评估性能，实验证明使用 WavLM 表征的 SVSNet + 相比基线模型有显著改进，将 WavLM 替换为其他 SFMs 时，SVSNet + 仍优于基线模型且表现出强大的泛化能力。