Lip reading, the process of interpreting silent speech from visual lip
movements, has gained rising attention for its wide range of realistic
applications. Deep learning approaches greatly improve current lip reading
systems. However, lip reading in cross-speaker scenarios where the speaker
identity changes, poses a challenging problem due to inter-speaker variability.
A well-trained lip reading system may perform poorly when handling a brand new
speaker. To learn a speaker-robust lip reading model, a key insight is to
reduce visual variations across speakers, avoiding the model overfitting to
specific speakers. In this work, in view of both input visual clues and latent
representations based on a hybrid CTC/attention architecture, we propose to
exploit the lip landmark-guided fine-grained visual clues instead of
frequently-used mouth-cropped images as input features, diminishing
speaker-specific appearance characteristics. Furthermore, a max-min mutual
information regularization approach is proposed to capture speaker-insensitive
latent representations. Experimental evaluations on public lip reading datasets
demonstrate the effectiveness of the proposed approach under the intra-speaker
and inter-speaker conditions.

本研究提出了一种训练鲁棒性的唇读模型的方法，通过利用唇部地标引导的细粒度视觉线索，减少与特定说话者相关的外观特征，并通过最大最小互信息正则化方法捕捉不受说话者影响的潜在表示。实验证明了该方法在说话者内部和跨说话者条件下的有效性。