Recently, large pre-trained multilingual speech models have shown potential
in scaling Automatic Speech Recognition (ASR) to many low-resource languages.
Some of these models employ language adapters in their formulation, which helps
to improve monolingual performance and avoids some of the drawbacks of
multi-lingual modeling on resource-rich languages. However, this formulation
restricts the usability of these models on code-switched speech, where two
languages are mixed together in the same utterance. In this work, we propose
ways to effectively fine-tune such models on code-switched speech, by
assimilating information from both language adapters at each language
adaptation point in the network. We also model code-switching as a sequence of
latent binary sequences that can be used to guide the flow of information from
each language adapter at the frame level. The proposed approaches are evaluated
on three code-switched datasets encompassing Arabic, Mandarin, and Hindi
languages paired with English, showing consistent improvements in
code-switching performance with at least 10\% absolute reduction in CER across
all test sets.

本文提出了在代码转换的语音中有效地微调大型预训练多语言语音模型的方法，并将代码转换建模为一系列潜在的二进制序列，以引导每个语言适配器在帧级别上的信息流，通过在阿拉伯语、普通话和印地语等多种语言与英语配对的数据集上的评估，显示出代码转换性能的持续改进，至少可以将语音识别错误率 (CER) 降低 10％。

多语音识别中的代码切换适配器适应

Adapting the adapters for code-switching in multilingual ASR

The conventional recipe for Automatic Speech Recognition (ASR) models is to
1) train multiple checkpoints on a training set while relying on a validation
set to prevent overfitting using early stopping and 2) average several last
checkpoints or that of the lowest validation losses to obtain the final model.
In this paper, we rethink and update the early stopping and checkpoint
averaging from the perspective of the bias-variance tradeoff. Theoretically,
the bias and variance represent the fitness and variability of a model and the
tradeoff of them determines the overall generalization error. But, it's
impractical to evaluate them precisely. As an alternative, we take the training
loss and validation loss as proxies of bias and variance and guide the early
stopping and checkpoint averaging using their tradeoff, namely an Approximated
Bias-Variance Tradeoff (ApproBiVT). When evaluating with advanced ASR models,
our recipe provides 2.5%-3.7% and 3.1%-4.6% CER reduction on the AISHELL-1 and
AISHELL-2, respectively.

在这篇论文中，我们从偏差和方差的权衡角度重新考虑并更新了早停和检查点平均值的方法，使用训练损失和验证损失作为偏差和方差的近似代理，并在高级 ASR 模型上验证时，我们的方法在 AISHELL-1 和 AISHELL-2 上分别降低了 2.5%-3.7% 和 3.1%-4.6% 的 CER。

ApproBiVT: 使用近似的偏差 - 方差折衷指导早停和检查点平均的先导 ASR 模型更好地泛化

ApproBiVT: Lead ASR Models to Generalize Better Using Approximated  Bias-Variance Tradeoff Guided Early Stopping and Checkpoint Averaging

Speech recognition technologies are gaining enormous popularity in various
industrial applications. However, building a good speech recognition system
usually requires large amounts of transcribed data, which is expensive to
collect. To tackle this problem, an unsupervised pre-training method called
Masked Predictive Coding is proposed, which can be applied for unsupervised
pre-training with Transformer based model. Experiments on HKUST show that using
the same training data, we can achieve CER 23.3%, exceeding the best end-to-end
model by over 0.2% absolute CER. With more pre-training data, we can further
reduce the CER to 21.0%, or a 11.8% relative CER reduction over baseline.

提出一种称为 Masked Predictive Coding 的无监督预训练方法，可用于 Transformer-based 模型的无监督预训练，其应用于 HKUST 的实验表明，使用相同的训练数据，我们可以实现 CER 为 23.3%，超过最佳的端到端模型 0.2% 的绝对 CER。使用更多的预训练数据，我们可以进一步将 CER 降至 21.0％，相对 CER 基线降低 11.8%。