Unsupervised speech representations have taken off, with benchmarks (SUPERB,
ZeroSpeech) demonstrating major progress on semi-supervised speech recognition,
speech synthesis, and speech-only language modelling. Inspiration comes from
the promise of ``discovering the phonemes'' of a language or a similar
low-bitrate encoding. However, one of the critical properties of phoneme
transcriptions is context-invariance: the phonetic context of a speech sound
can have massive influence on the way it is pronounced, while the text remains
stable. This is what allows tokens of the same word to have the same
transcriptions -- key to language understanding. Current benchmarks do not
measure context-invariance. We develop a new version of the ZeroSpeech ABX
benchmark that measures context-invariance, and apply it to recent
self-supervised representations. We demonstrate that the context-independence
of representations is predictive of the stability of word-level
representations. We suggest research concentrate on improving
context-independence of self-supervised and unsupervised representations.

本文介绍了语音或文本的不变性对于理解语言至关重要，提出了一种新的 ZeroSpeech ABX 基准测试方法来测试表示的上下文独立性，并建议将重点放在改善自监督和无监督表示的上下文独立性上。

评估无监督语音表示中的上下文不变性

Evaluating context-invariance in unsupervised speech representations

We present a method for visually-grounded spoken term discovery. After
training either a HuBERT or wav2vec2.0 model to associate spoken captions with
natural images, we show that powerful word segmentation and clustering
capability emerges within the model's self-attention heads. Our experiments
reveal that this ability is not present to nearly the same extent in the base
HuBERT and wav2vec2.0 models, suggesting that the visual grounding task is a
crucial component of the word discovery capability we observe. We also evaluate
our method on the Buckeye word segmentation and ZeroSpeech spoken term
discovery tasks, where we outperform all currently published methods on several
metrics. Code and model weights are available at
this https URL.

本文提出了一种可视化感知的口语术语探测方法，通过对自注意力头的训练与分析发现，在自然图像与口语字幕关联的模型中具有强大的单词分割和聚类能力，并在 Buckeye 单词分割和 ZeroSpeech 口语术语探测任务上超过了所有已发表的现有方法。