The first Chinese Continuous Visual Speech Recognition Challenge aimed to
probe the performance of Large Vocabulary Continuous Visual Speech Recognition
(LVC-VSR) on two tasks: (1) Single-speaker VSR for a particular speaker and (2)
Multi-speaker VSR for a set of registered speakers. The challenge yielded
highly successful results, with the best submission significantly outperforming
the baseline, particularly in the single-speaker task. This paper
comprehensively reviews the challenge, encompassing the data profile, task
specifications, and baseline system construction. It also summarises the
representative techniques employed by the submitted systems, highlighting the
most effective approaches. Additional information and resources about this
challenge can be accessed through the official website at
this http URL

第一次的中国连续视觉语音识别挑战旨在评估大词汇量连续视觉语音识别在两个任务上的性能：（1）针对特定说话人的单一说话人语音识别和（2）针对一组已注册说话人的多说话人语音识别。此次挑战取得了非常成功的结果，最佳提交结果明显优于基准，尤其是在单一说话人任务中。本文全面回顾了这个挑战，包括数据概况、任务规范和基准系统的构建。同时，它总结了参赛系统所采用的代表性技术，并突出了最有效的方法。有关此挑战的更多信息和资源可通过官方网站访问此 http URL。

CNVSRC 2023：首届中国连续视觉语音识别挑战

CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition  Challenge

This study describes our system for Task 1 Single-speaker Visual Speech
Recognition (VSR) fixed track in the Chinese Continuous Visual Speech
Recognition Challenge (CNVSRC) 2023. Specifically, we use intermediate
connectionist temporal classification (Inter CTC) residual modules to relax the
conditional independence assumption of CTC in our model. Then we use a
bi-transformer decoder to enable the model to capture both past and future
contextual information. In addition, we use Chinese characters as the modeling
units to improve the recognition accuracy of our model. Finally, we use a
recurrent neural network language model (RNNLM) for shallow fusion in the
inference stage. Experiments show that our system achieves a character error
rate (CER) of 38.09% on the Eval set which reaches a relative CER reduction of
21.63% over the official baseline, and obtains a second place in the challenge.

这项研究描述了我们在 2023 年中国持续视觉语音识别挑战赛中针对任务 1 单说话者视觉语音识别（VSR）固定轨道的系统。具体来说，我们在模型中使用中间连接主义时间分类（Inter CTC）残差模块来放宽 CTC 的条件独立性假设，然后使用双变压器解码器使模型能够捕捉过去和未来的上下文信息。此外，我们使用汉字作为建模单元来提高模型的识别准确性。最后，在推断阶段，我们使用了递归神经网络语言模型（RNNLM）进行浅层融合。实验表明，我们的系统在评估集上的字符错误率（CER）为 38.09%，相对于官方基准线减少了 21.63%，并在挑战赛中获得了第二名。