In this paper, we propose two novel augmentation methods 1) audio-language
MixGen (AL-MixGen) and 2) multi-level test-time augmentation (Multi-TTA) for
audio-language learning. Inspired by MixGen, which is originally applied to
vision-language learning, we introduce an augmentation method for the
audio-language domain. We also explore the impact of test-time augmentations
and present Multi-TTA which generalizes test-time augmentation over multiple
layers of a deep learning model. Incorporating AL-MixGen and Multi-TTA into the
baseline achieves 47.5 SPIDEr on audio captioning, which is an +18.2% over the
baseline and outperforms the state-of-the-art approach with a 5x smaller model.
In audio-text retrieval, the proposed methods surpass the baseline performance
as well.

本文提出了两种新的数据增强方法：1）“音频 - 语言合成增强”（AL-MixGen）和 2）“多层测试时间增强”（Multi-TTA）用于音频和语言的学习；其中 Multi-TTA 可以将测试时间增强一般化到深度学习模型的多个层面。将这些方法引入到基线中，对于音频字幕生成任务可以获得 47.5 SPIDEr 的性能，相比基线提高了 18.2％，模型参数也小了 5 倍，同时在音频 - 文本检索中也超过了基线性能。