We investigate the performance on phoneme categorization and phoneme and word
segmentation of several self-supervised learning (SSL) methods based on
Contrastive Predictive Coding (CPC). Our experiments show that with the
existing algorithms there is a trade off between categorization and
segmentation performance. We investigate the source of this conflict and
conclude that the use of context building networks, albeit necessary for
superior performance on categorization tasks, harms segmentation performance by
causing a temporal shift on the learned representations. Aiming to bridge this
gap, we take inspiration from the leading approach on segmentation, which
simultaneously models the speech signal at the frame and phoneme level, and
incorporate multi-level modelling into Aligned CPC (ACPC), a variation of CPC
which exhibits the best performance on categorization tasks. Our multi-level
ACPC (mACPC) improves in all categorization metrics and achieves
state-of-the-art performance in word segmentation.

本文研究了基于对比预测编码 (CPC) 的自监督学习 (SSL) 方法在音素分类和音素、单词分割方面的性能。结果表明，现有算法在分类和分割性能上存在平衡。为了弥合这种差距，我们借鉴了在分割方面较好的方法，并将多级建模方法整合到 CPC 的改进版本 Aligned CPC (ACPC) 中，提高了在所有分类指标上的性能，并在单词分割方面取得了最先进的性能。