Unsupervised automatic speech recognition (ASR) aims to learn the mapping
between the speech signal and its corresponding textual transcription without
the supervision of paired speech-text data. A word/phoneme in the speech signal
is represented by a segment of speech signal with variable length and unknown
boundary, and this segmental structure makes learning the mapping between
speech and text challenging, especially without paired data. In this paper, we
propose REBORN, Reinforcement-Learned Boundary Segmentation with Iterative
Training for Unsupervised ASR. REBORN alternates between (1) training a
segmentation model that predicts the boundaries of the segmental structures in
speech signals and (2) training the phoneme prediction model, whose input is a
segmental structure segmented by the segmentation model, to predict a phoneme
transcription. Since supervised data for training the segmentation model is not
available, we use reinforcement learning to train the segmentation model to
favor segmentations that yield phoneme sequence predictions with a lower
perplexity. We conduct extensive experiments and find that under the same
setting, REBORN outperforms all prior unsupervised ASR models on LibriSpeech,
TIMIT, and five non-English languages in Multilingual LibriSpeech. We
comprehensively analyze why the boundaries learned by REBORN improve the
unsupervised ASR performance.

无监督自动语音识别（ASR）的关键问题是在没有成对的语音 - 文本数据监督的情况下，学习语音信号与其相应的文本转录之间的映射。本文提出了一种名为 REBORN 的方法，用于无监督 ASR，通过交替训练分割模型和音素预测模型来学习语音和文本之间的映射，以提高无监督 ASR 的性能。