Speech restoration aims at restoring high quality speech in the presence of a
diverse set of distortions. Although several deep learning paradigms have been
studied for this task, the power of the recently emerging language models has
not been fully explored. In this paper, we propose MaskSR, a masked language
model capable of restoring full-band 44.1 kHz speech jointly considering noise,
reverb, clipping, and low bandwidth. MaskSR works with discrete acoustic tokens
extracted using a pre-trained neural codec. During training, MaskSR is
optimized to predict randomly masked tokens extracted from the high quality
target speech, conditioned on the corrupted speech with various distortions.
During inference, MaskSR reconstructs the target speech tokens with efficient
iterative sampling. Extensive experiments show that MaskSR obtains competitive
results on both the full-band speech restoration task and also on sub-tasks
compared with a wide range of models.

语音恢复是在各种失真的情况下恢复高品质语音的目标。本文提出了一种名为 MaskSR 的掩码语言模型，能够联合考虑噪声、混响、剪切和低带宽来恢复全频 44.1 kHz 的语音。MaskSR 利用预训练的神经编解码器提取离散声学令牌。在训练过程中，MaskSR 被优化为根据带有各种失真的损坏语音，预测从高品质目标语音中随机屏蔽的令牌。在推断过程中，MaskSR 通过高效的迭代采样重建目标语音令牌。大量实验证明，与各种模型相比，MaskSR 在全频语音恢复任务和子任务上都取得了竞争力的结果。