While federated learning (FL) has recently emerged as a promising approach to
train machine learning models, it is limited to only preliminary explorations
in the domain of automatic speech recognition (ASR). Moreover, FL does not
inherently guarantee user privacy and requires the use of differential privacy
(DP) for robust privacy guarantees. However, we are not aware of prior work on
applying DP to FL for ASR. In this paper, we aim to bridge this research gap by
formulating an ASR benchmark for FL with DP and establishing the first
baselines. First, we extend the existing research on FL for ASR by exploring
different aspects of recent $\textit{large end-to-end transformer models}$:
architecture design, seed models, data heterogeneity, domain shift, and impact
of cohort size. With a $\textit{practical}$ number of central aggregations we
are able to train $\textbf{FL models}$ that are \textbf{nearly optimal} even
with heterogeneous data, a seed model from another domain, or no pre-trained
seed model. Second, we apply DP to FL for ASR, which is non-trivial since DP
noise severely affects model training, especially for large transformer models,
due to highly imbalanced gradients in the attention block. We counteract the
adverse effect of DP noise by reviving per-layer clipping and explaining why
its effect is more apparent in our case than in the prior work. Remarkably, we
achieve user-level ($7.2$, $10^{-9}$)-$\textbf{DP}$ (resp. ($4.5$,
$10^{-9}$)-$\textbf{DP}$) with a 1.3% (resp. 4.6%) absolute drop in the word
error rate for extrapolation to high (resp. low) population scale for
$\textbf{FL with DP in ASR}$.

新论文提出了在自动语音识别中利用差分隐私实现联邦学习的模型，该模型通过对大型端到端变换器模型的架构设计、种子模型、数据异构性、领域转换和队友规模的影响等因素进行研究，实现了几乎最佳的联邦学习模型，同时应用差分隐私还能在用户级别保护用户隐私。