We introduce a new unsupervised task, spoken language modeling: the learning
of linguistic representations from raw audio signals without any labels, along
with the Zero Resource Speech Benchmark 2021: a suite of 4 black-box, zero-shot
metrics probing for the quality of the learned models at 4 linguistic levels:
phonetics, lexicon, syntax and semantics. We present the results and analyses
of a composite baseline made of the concatenation of three unsupervised
systems: self-supervised contrastive representation learning (CPC), clustering
(k-means) and language modeling (LSTM or BERT). The language models learn on
the basis of the pseudo-text derived from clustering the learned
representations. This simple pipeline shows better than chance performance on
all four metrics, demonstrating the feasibility of spoken language modeling
from raw speech. It also yields worse performance compared to text-based
'topline' systems trained on the same data, delineating the space to be
explored by more sophisticated end-to-end models.

本文介绍了一项新的无监督任务：口语语言建模并给出了 Zero Resource Speech Benchmark 2021，使用 4 个黑盒子零炮弹指标对 4 个语言层面的学习模型的质量进行测量，包括语音、词汇、语法和语义，并基于 CPC、聚类和语言建模的三个无监督系统的复合基线的结果和分析。该语言建模基于聚类学习产生的伪文本，表明了从原始语音中建立口语语言建模的可行性，但这种简单管道与文本为基础的 “顶线” 系统相比，性能较差，需要使用更复杂的端到端模型来探索。