In this paper, we propose SemanticAC, a semantics-assisted framework for
Audio Classification to better leverage the semantic information. Unlike
conventional audio classification methods that treat class labels as discrete
vectors, we employ a language model to extract abundant semantics from labels
and optimize the semantic consistency between audio signals and their labels.
We verify that simple textual information from labels and advanced pretraining
models enable more abundant semantic supervision for better performance.
Specifically, we design a text encoder to capture the semantic information from
the text extension of labels. Then we map the audio signals to align with the
semantics of corresponding class labels via an audio encoder and a similarity
calculation module so as to enforce the semantic consistency. Extensive
experiments on two audio datasets, ESC-50 and US8K demonstrate that our
proposed method consistently outperforms the compared audio classification
methods.

本文提出了一个语义辅助的音频分类框架 SemanticAC，它通过利用标签中的语义信息，保证音频信号与标签之间的语义一致性，从而实现更好的性能。通过在 ESC-50 和 US8K 两个音频数据集上进行广泛实验，验证了该方法能够持续优于其他音频分类方法。