We introduce "Unspeech" embeddings, which are based on unsupervised learning
of context feature representations for spoken language. The embeddings were
trained on up to 9500 hours of crawled English speech data without
transcriptions or speaker information, by using a straightforward learning
objective based on context and non-context discrimination with negative
sampling. We use a Siamese convolutional neural network architecture to train
Unspeech embeddings and evaluate them on speaker comparison, utterance
clustering and as a context feature in TDNN-HMM acoustic models trained on
TED-LIUM, comparing it to i-vector baselines. Particularly decoding
out-of-domain speech data from the recently released Common Voice corpus shows
consistent WER reductions. We release our source code and pre-trained Unspeech
models under a permissive open source license.

本文介绍了基于无监督学习的 Unspeech 嵌入表示，是对口语语言上下文特征表示的学习，并使用孪生卷积神经网络构建该模型，应用于演讲数据的识别以及 TED-LIUM 的 TDNN-HMM 声学模型中，可用于讲话人比较、话语聚类和语音识别等领域，并公开了源代码和已经预训练的 Unspeech 模型。