We investigate the performance on phoneme categorization and phoneme and word
segmentation of several self-supervised learning (SSL) methods based on
Contrastive Predictive Coding (CPC). Our experiments show that with the
existing algorithms there is a trade off between categorization and
segmentation performance. We investigate the source of this conflict and
conclude that the use of context building networks, albeit necessary for
superior performance on categorization tasks, harms segmentation performance by
causing a temporal shift on the learned representations. Aiming to bridge this
gap, we take inspiration from the leading approach on segmentation, which
simultaneously models the speech signal at the frame and phoneme level, and
incorporate multi-level modelling into Aligned CPC (ACPC), a variation of CPC
which exhibits the best performance on categorization tasks. Our multi-level
ACPC (mACPC) improves in all categorization metrics and achieves
state-of-the-art performance in word segmentation.

本文研究了基于对比预测编码 (CPC) 的自监督学习 (SSL) 方法在音素分类和音素、单词分割方面的性能。结果表明，现有算法在分类和分割性能上存在平衡。为了弥合这种差距，我们借鉴了在分割方面较好的方法，并将多级建模方法整合到 CPC 的改进版本 Aligned CPC (ACPC) 中，提高了在所有分类指标上的性能，并在单词分割方面取得了最先进的性能。

无监督分割和分类音素和单词的对比预测策略

Contrastive prediction strategies for unsupervised segmentation and  categorization of phonemes and words

Automatic detection of phoneme or word-like units is one of the core
objectives in zero-resource speech processing. Recent attempts employ
self-supervised training methods, such as contrastive predictive coding (CPC),
where the next frame is predicted given past context. However, CPC only looks
at the audio signal's frame-level structure. We overcome this limitation with a
segmental contrastive predictive coding (SCPC) framework that can model the
signal structure at a higher level e.g. at the phoneme level. In this
framework, a convolutional neural network learns frame-level representation
from the raw waveform via noise-contrastive estimation (NCE). A differentiable
boundary detector finds variable-length segments, which are then used to
optimize a segment encoder via NCE to learn segment representations. The
differentiable boundary detector allows us to train frame-level and
segment-level encoders jointly. Typically, phoneme and word segmentation are
treated as separate tasks. We unify them and experimentally show that our
single model outperforms existing phoneme and word segmentation methods on
TIMIT and Buckeye datasets. We analyze the impact of boundary threshold and
when is the right time to include the segmental loss in the learning process.

本文提出了一个用于零资源语音处理的新的模型，称为分段对比预测编码，能够对音频信号的帧级和更高级别的分组进行编码，并将此模型用于音素和单词分割，结果在 TIMIT 和 Buckeye 数据集上显著优于现有方法。