This paper proposes a novel, resource-efficient approach to Visual Speech
Recognition (VSR) leveraging speech representations produced by any trained
Automatic Speech Recognition (ASR) model. Moving away from the
resource-intensive trends prevalent in recent literature, our method distills
knowledge from a trained Conformer-based ASR model, achieving competitive
performance on standard VSR benchmarks with significantly less resource
utilization. Using unlabeled audio-visual data only, our baseline model
achieves a word error rate (WER) of 47.4% and 54.7% on the LRS2 and LRS3 test
benchmarks, respectively. After fine-tuning the model with limited labeled
data, the word error rate reduces to 35% (LRS2) and 45.7% (LRS3). Our model can
be trained on a single consumer-grade GPU within a few days and is capable of
performing real-time end-to-end VSR on dated hardware, suggesting a path
towards more accessible and resource-efficient VSR methodologies.

该论文提出了一种新颖的资源高效方法，利用任何训练过的自动语音识别（ASR）模型生成的语音表示来进行视觉语音识别（VSR）。通过远离最近文献中流行的资源密集型趋势，我们的方法从已训练的基于 Conformer 的 ASR 模型中提取知识，在标准的 VSR 基准测试中以极少的资源利用实现竞争性的性能。仅使用未标记的音频 - 视觉数据，我们的基准模型在 LRS2 和 LRS3 测试基准上分别达到了 47.4% 和 54.7% 的词错误率（WER）。在有限标记数据的微调之后，词错误率降至 35%（LRS2）和 45.7%（LRS3）。我们的模型可以在几天内在单个消费级 GPU 上进行训练，并能够在老旧硬件上实时进行端到端的 VSR，为实现更易于获取和高效利用资源的 VSR 方法提供了一个路径。