The goal of universal audio representation learning is to obtain foundational
models that can be used for a variety of downstream tasks involving speech,
music or environmental sounds. To approach this problem, methods inspired by
self-supervised models from NLP, like BERT, are often used and adapted to
audio. These models rely on the discrete nature of text, hence adopting this
type of approach for audio processing requires either a change in the learning
objective or mapping the audio signal to a set of discrete classes. In this
work, we explore the use of EnCodec, a neural audio codec, to generate discrete
targets for learning an universal audio model based on a masked autoencoder
(MAE). We evaluate this approach, which we call EncodecMAE, on a wide range of
audio tasks spanning speech, music and environmental sounds, achieving
performances comparable or better than leading audio representation models.

本研究探讨了使用神经音频编解码器 EnCodec 生成发音、音乐和环境声音的离散目标，以学习基于遮蔽自动编码器的通用音频模型 EncodecMAE，并在广泛的音频任务中取得了与领先的音频表示模型相媲美或更好的性能。