Multi-modal learning in the audio-language domain has seen significant
advancements in recent years. However, audio-language learning faces challenges
due to limited and lower-quality data compared to image-language tasks.
Existing audio-language datasets are notably smaller, and manual labeling is
hindered by the need to listen to entire audio clips for accurate labeling.
Our method systematically generates audio-caption pairs by augmenting audio
clips with natural language labels and corresponding audio signal processing
operations. Leveraging a Large Language Model, we generate descriptions of
augmented audio clips with a prompt template. This scalable method produces
AudioSetMix, a high-quality training dataset for text-and-audio related models.
Integration of our dataset improves models performance on benchmarks by
providing diversified and better-aligned examples. Notably, our dataset
addresses the absence of modifiers (adjectives and adverbs) in existing
datasets. By enabling models to learn these concepts, and generating hard
negative examples during training, we achieve state-of-the-art performance on
multiple benchmarks.

我们通过增加自然语言标签和相应的音频信号处理操作，使用大型语言模型提供了一个高质量的训练数据集，该数据集在文本和音频相关模型的基准测试中提供了多样化且更好对齐的示例，从而改善了模型的性能。