In this work, we devise a parameter-efficient solution to bring differential
privacy (DP) guarantees into adaptation of a cross-lingual speech classifier.
We investigate a new frozen pre-trained adaptation framework for DP-preserving
speech modeling without full model fine-tuning. First, we introduce a noisy
teacher-student ensemble into a conventional adaptation scheme leveraging a
frozen pre-trained acoustic model and attain superior performance than DP-based
stochastic gradient descent (DPSGD). Next, we insert residual adapters (RA)
between layers of the frozen pre-trained acoustic model. The RAs reduce
training cost and time significantly with a negligible performance drop.
Evaluated on the open-access Multilingual Spoken Words (MLSW) dataset, our
solution reduces the number of trainable parameters by 97.5% using the RAs with
only a 4% performance drop with respect to fine-tuning the cross-lingual speech
classifier while preserving DP guarantees.

通过噪声教师学生模型和残余适配器的引入，本研究提出了一种参数高效的解决方案，实现跨语言语音分类器的差分隐私保护，通过在训练之前进行一次性预训练，将模型的参数数量减少了 97.5%，并且只有 4% 的性能下降。

差分隐私适配器用于参数高效声学建模

Differentially Private Adapters for Parameter Efficient Acoustic  Modeling

Adapting a neural text-to-speech (TTS) model to a target speaker typically
involves fine-tuning most if not all of the parameters of a pretrained
multi-speaker backbone model. However, serving hundreds of fine-tuned neural
TTS models is expensive as each of them requires significant footprint and
separate computational resources (e.g., accelerators, memory). To scale speaker
adapted neural TTS voices to hundreds of speakers while preserving the
naturalness and speaker similarity, this paper proposes a parameter-efficient
few-shot speaker adaptation, where the backbone model is augmented with
trainable lightweight modules called residual adapters. This architecture
allows the backbone model to be shared across different target speakers.
Experimental results show that the proposed approach can achieve competitive
naturalness and speaker similarity compared to the full fine-tuning approaches,
while requiring only $\sim$0.1% of the backbone model parameters for each
speaker.

本研究提出了一种参数高效的少样本说话人自适应方法，通过使用可训练轻量级模块 (residual adapters) 来增加骨干模型 (backbone model) 实现对各种目标说话人的共享，可以在保持语音自然性和说话人相似性的同时仅使用上亿模型参数的 0.1％。

用于少样本文本转语音说话人自适应的剩余适配器

Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptation

Continual learning in task-oriented dialogue systems can allow us to add new
domains and functionalities through time without incurring the high cost of a
whole system retraining. In this paper, we propose a continual learning
benchmark for task-oriented dialogue systems with 37 domains to be learned
continuously in four settings, such as intent recognition, state tracking,
natural language generation, and end-to-end. Moreover, we implement and compare
multiple existing continual learning baselines, and we propose a simple yet
effective architectural method based on residual adapters. Our experiments
demonstrate that the proposed architectural method and a simple replay-based
strategy perform comparably well but they both achieve inferior performance to
the multi-task learning baseline, in where all the data are shown at once,
showing that continual learning in task-oriented dialogue systems is a
challenging task. Furthermore, we reveal several trade-offs between different
continual learning methods in term of parameter usage and memory size, which
are important in the design of a task-oriented dialogue system. The proposed
benchmark is released together with several baselines to promote more research
in this direction.

本文提出了一个能够让我们通过时间添加新的领域和功能而不需要付出整个系统重新训练的任务导向对话系统的连续学习基准。我们在四个不同的设置下连续地学习了 37 个领域，包括意图识别、状态跟踪、自然语言生成和端到端。此外，我们实现并比较了多个已有的持续学习基线，并提出了一种简单而有效的基于残余适配器的结构方法。我们的实验证明，所提出的结构方法和简单的回放策略表现相当好，但是它们都比多任务学习基线表现差，后者将所有数据一次性展示出来，这表明对话系统的连续学习是一项具有挑战性的任务。此外，我们还揭示了在参数使用和内存大小方面不同连续学习方法之间的几个权衡，这对任务导向对话系统的设计非常重要。我们还发布了所提出的基准测试以及几个基线，以推动更多的研究方向。