Machine lipreading is a special type of automatic speech recognition (ASR)
which transcribes human speech by visually interpreting the movement of related
face regions including lips, face, and tongue. Recently, deep neural network
based lipreading methods show great potential and have exceeded the accuracy of
experienced human lipreaders in some benchmark datasets. However, lipreading is
still far from being solved, and existing methods tend to have high error rates
on the wild data. In this paper, we propose LCANet, an end-to-end deep neural
network based lipreading system. LCANet encodes input video frames using a
stacked 3D convolutional neural network (CNN), highway network and
bidirectional GRU network. The encoder effectively captures both short-term and
long-term spatio-temporal information. More importantly, LCANet incorporates a
cascaded attention-CTC decoder to generate output texts. By cascading CTC with
attention, it partially eliminates the defect of the conditional independence
assumption of CTC within the hidden neural layers, and this yields notably
performance improvement as well as faster convergence. The experimental results
show the proposed system achieves a 1.3% CER and 3.0% WER on the GRID corpus
database, leading to a 12.3% improvement compared to the state-of-the-art
methods.

本文提出了一种基于端到端深度神经网络的 LCANet 口型识别系统，利用 3D CNN、highway network 和双向 GRU network 来捕捉短期和长期的空时信息，并采用连续的注意力 - CTC 解码器来生成输出文字，实验结果表明，与现有的最先进方法相比，所提出的系统在 GRID 语料库上取得了 1.3% 的 CER 和 3.0% 的 WER，有 12.3% 的改进。