Learning rich visual representations using contrastive self-supervised
learning has been extremely successful. However, it is still a major question
whether we could use a similar approach to learn superior auditory
representations. In this paper, we expand on prior work (SimCLR) to learn
better auditory representations. We (1) introduce various data augmentations
suitable for auditory data and evaluate their impact on predictive performance,
(2) show that training with time-frequency audio features substantially
improves the quality of the learned representations compared to raw signals,
and (3) demonstrate that training with both supervised and contrastive losses
simultaneously improves the learned representations compared to self-supervised
pre-training followed by supervised fine-tuning. We illustrate that by
combining all these methods and with substantially less labeled data, our
framework (CLAR) achieves significant improvement on prediction performance
compared to supervised approach. Moreover, compared to self-supervised
approach, our framework converges faster with significantly better
representations.

本文基于 SimCLR 的前期工作，提出了适用于音频数据的各种数据增强方案，并调查了它们对预测性能的影响，同时证明了采用时频音频特征训练，在监督和对比损失同时约束下的模型可以获得优秀的音频表示。在少量标注数据的情况下，该方法明显地改善了预测性能，同时比自监督训练更快地收敛并具有更好的表示能力。