Despite its widespread adoption as the prominent neural architecture, the
Transformer has spurred several independent lines of work to address its
limitations. One such approach is selective state space models, which have
demonstrated promising results for language modelling. However, their
feasibility for learning self-supervised, general-purpose audio representations
is yet to be investigated. This work proposes Audio Mamba, a selective state
space model for learning general-purpose audio representations from randomly
masked spectrogram patches through self-supervision. Empirical results on ten
diverse audio recognition downstream tasks show that the proposed models,
pretrained on the AudioSet dataset, consistently outperform comparable
self-supervised audio spectrogram transformer (SSAST) baselines by a
considerable margin and demonstrate better performance in dataset size,
sequence length and model size comparisons.

提出了一种名为 Audio Mamba 的选择性状态空间模型，通过自我监督学习从随机遮罩的频谱图补丁中学习通用的音频表示。实证结果表明，预训练于 AudioSet 数据集的该模型在十个不同的音频识别任务中持续优于可比较的自我监督音频谱图 Transformer (SSAST) 基线，表现出更好的数据集大小、序列长度和模型大小的对比性能。