We propose a novel method that combines CycleGAN and inter-domain losses for
semi-supervised end-to-end automatic speech recognition. Inter-domain loss
targets the extraction of an intermediate shared representation of speech and
text inputs using a shared network. CycleGAN uses cycle-consistent loss and the
identity mapping loss to preserve relevant characteristics of the input feature
after converting from one domain to another. As such, both approaches are
suitable to train end-to-end models on unpaired speech-text inputs. In this
paper, we exploit the advantages from both inter-domain loss and CycleGAN to
achieve better shared representation of unpaired speech and text inputs and
thus improve the speech-to-text mapping. Our experimental results on the WSJ
eval92 and Voxforge (non English) show 8~8.5% character error rate reduction
over the baseline, and the results on LibriSpeech test_clean also show
noticeable improvement.

本研究提出了一种结合 CycleGAN 和域间损失的新方法，用于半监督的端到端自动语音识别，利用这种方法可以在无配对的语音文本输入上进行训练，并取得了明显的性能提升。