Previous studies have confirmed the effectiveness of incorporating visual
information into speech enhancement (SE) systems. Despite improved denoising
performance, two problems may be encountered when implementing an audio-visual
SE (AVSE) system: (1) additional processing costs are incurred to incorporate
visual input and (2) the use of face or lip images may cause privacy problems.
In this study, we propose a Lite AVSE (LAVSE) system to address these problems.
The system includes two visual data compression techniques and removes the
visual feature extraction network from the training model, yielding better
online computation efficiency. Our experimental results indicate that the
proposed LAVSE system can provide notably better performance than an audio-only
SE system with a similar number of model parameters. In addition, the
experimental results confirm the effectiveness of the two techniques for visual
data compression.

本研究提出了一种 “轻量级视听语音增强” 系统（LAVSE），该系统通过采用两种视觉数据压缩技术和去除训练模型中的人脸或唇部图像的特征提取网络，以达到更好的在线计算效率和更好的性能，从而在保障隐私的前提下，实现了比仅使用音频的增强系统更出色的性能表现。