This paper proposes InterAug: a novel training method for CTC-based ASR using augmented intermediate representations for conditioning. The proposed method exploits the conditioning framework of self-conditioned CTC to train robust models by conditioning with "noisy" intermediate predictions. During the training, intermediate predictions are changed to incorrect intermediate predictions, and fed into the next layer for conditioning. The subsequent layers are trained to correct the incorrect intermediate predictions with the intermediate losses. By repeating the augmentation and the correction, iterative refinements, which generally require a special decoder, can be realized only with the audio encoder. To produce noisy intermediate predictions, we also introduce new augmentation: intermediate feature space augmentation and intermediate token space augmentation that are designed to simulate typical errors. The combination of the proposed InterAug framework with new augmentation allows explicit training of the robust audio encoders. In experiments using augmentations simulating deletion, insertion, and substitution error, we confirmed that the trained model acquires robustness to each error, boosting the speech recognition performance of the strong self-conditioned CTC baseline.

本论文提出了InterAug:一种使用增强的中间表示进行条件化的基于CTC的ASR的新训练方法。所提出的方法充分利用了自身条件付CTC的调节框架，通过在中间预测结果的基础上进行调节来训练鲁棒模型，从而实现了迭代式精炼。在使用模拟删除、插入和替换误差的增强进行的实验中，验证了训练模型对每种误差都具有强韧性的性能，提高了强自我条件付CTC基线的语音识别性能。

InterAug: CTC-based ASR中增强噪声中间预测