Recent work in spoken language modeling shows the possibility of learning a
language unsupervisedly from raw audio without any text labels. The approach
relies first on transforming the audio into a sequence of discrete units (or
pseudo-text) and then training a language model directly on such pseudo-text.
Is such a discrete bottleneck necessary, potentially introducing irreversible
errors in the encoding of the speech signal, or could we learn a language model
without discrete units at all? In this work, we study the role of discrete
versus continuous representations in spoken language modeling. We show that
discretization is indeed essential for good results in spoken language
modeling. We show that discretization removes linguistically irrelevant
information from the continuous features, helping to improve language modeling
performances. On the basis of this study, we train a language model on the
discrete units of the HuBERT features, reaching new state-of-the-art results in
the lexical, syntactic and semantic metrics of the Zero Resource Speech
Challenge 2021 (Track 1 - Speech Only).

本文研究了在语音语言建模中，离散与连续表示的作用，结论表明离散化具有重要作用，并在 HuBERT 特征上训练了语言模型，在 Zero Resource Speech Challenge 2021 中实现了最新的语言词汇、句法和语义方面的最佳表现。