This paper presents the development of a speech synthesis system for the
LIMMITS'24 Challenge, focusing primarily on Track 2. The objective of the
challenge is to establish a multi-speaker, multi-lingual Indic Text-to-Speech
system with voice cloning capabilities, covering seven Indian languages with
both male and female speakers. The system was trained using challenge data and
fine-tuned for few-shot voice cloning on target speakers. Evaluation included
both mono-lingual and cross-lingual synthesis across all seven languages, with
subjective tests assessing naturalness and speaker similarity. Our system uses
the VITS2 architecture, augmented with a multi-lingual ID and a BERT model to
enhance contextual language comprehension. In Track 1, where no additional data
usage was permitted, our model achieved a Speaker Similarity score of 4.02. In
Track 2, which allowed the use of extra data, it attained a Speaker Similarity
score of 4.17.

本文介绍了一个语音合成系统的开发，用于 LIMMITS'24 挑战赛，主要关注第二阶段。该系统的目标是建立一个具有语音克隆功能的多说话人、多语言的印度文本到语音系统，涵盖了七种印度语言和男女两性的发言人。该系统通过使用挑战数据进行训练，并针对目标发言人进行少样本语音克隆进行了微调。评估包括对所有七种语言进行单语和跨语言综合的主观测试，评估自然度和发言人相似性。我们的系统使用了 VITS2 架构，结合多语言 ID 和 BERT 模型来增强上下文语言理解。在第一阶段，不允许使用额外数据的情况下，我们的模型的发言人相似性得分为 4.02。而在允许使用额外数据的第二阶段，得分为 4.17。

一种基于 VITS2 的多说话人多语言语音克隆系统，用于 limmits2024 挑战

A multi-speaker multi-lingual voice cloning system based on vits2 for  limmits 2024 challenge

We present MunTTS, an end-to-end text-to-speech (TTS) system specifically for
Mundari, a low-resource Indian language of the Austo-Asiatic family. Our work
addresses the gap in linguistic technology for underrepresented languages by
collecting and processing data to build a speech synthesis system. We begin our
study by gathering a substantial dataset of Mundari text and speech and train
end-to-end speech models. We also delve into the methods used for training our
models, ensuring they are efficient and effective despite the data constraints.
We evaluate our system with native speakers and objective metrics,
demonstrating its potential as a tool for preserving and promoting the Mundari
language in the digital age.

针对孟达里语，这篇研究论文介绍了 MunTTS，一个专门用于文本到语音（TTS）的端到端系统，通过收集和处理数据来构建一个语音合成系统，以填补低资源印度语言在语言技术方面的不足，并通过与母语者和客观指标的评估展示其在数字时代中作为保护和推广孟达里语的工具的潜力。

MunTTS: 一个 Mundari 语文到语音系统

MunTTS: A Text-to-Speech System for Mundari

Personalizing a speech synthesis system is a highly desired application,
where the system can generate speech with the user's voice with rare enrolled
recordings. There are two main approaches to build such a system in recent
works: speaker adaptation and speaker encoding. On the one hand, speaker
adaptation methods fine-tune a trained multi-speaker text-to-speech (TTS) model
with few enrolled samples. However, they require at least thousands of
fine-tuning steps for high-quality adaptation, making it hard to apply on
devices. On the other hand, speaker encoding methods encode enrollment
utterances into a speaker embedding. The trained TTS model can synthesize the
user's speech conditioned on the corresponding speaker embedding. Nevertheless,
the speaker encoder suffers from the generalization gap between the seen and
unseen speakers.
In this paper, we propose applying a meta-learning algorithm to the speaker
adaptation method. More specifically, we use Model Agnostic Meta-Learning
(MAML) as the training algorithm of a multi-speaker TTS model, which aims to
find a great meta-initialization to adapt the model to any few-shot speaker
adaptation tasks quickly. Therefore, we can also adapt the meta-trained TTS
model to unseen speakers efficiently. Our experiments compare the proposed
method (Meta-TTS) with two baselines: a speaker adaptation method baseline and
a speaker encoding method baseline. The evaluation results show that Meta-TTS
can synthesize high speaker-similarity speech from few enrollment samples with
fewer adaptation steps than the speaker adaptation baseline and outperforms the
speaker encoding baseline under the same training scheme. When the speaker
encoder of the baseline is pre-trained with extra 8371 speakers of data,
Meta-TTS can still outperform the baseline on LibriTTS dataset and achieve
comparable results on VCTK dataset.

本文提出了一种使用元学习算法的多说话人文本转语音（Meta-TTS）模型，通过少量的适应步骤，可以生成具有高语音相似度的语音合成，并且比已有的说话人适应方法和说话人编码方法更为高效。