This work presents a text-to-audio-retrieval system based on pre-trained text
and spectrogram transformers. Our method projects recordings and textual
descriptions into a shared audio-caption space in which related examples from
different modalities are close. Through a systematic analysis, we examine how
each component of the system influences retrieval performance. As a result, we
identify two key components that play a crucial role in driving performance:
the self-attention-based audio encoder for audio embedding and the utilization
of additional human-generated and synthetic data sets during pre-training. We
further experimented with augmenting ClothoV2 captions with available keywords
to increase their variety; however, this only led to marginal improvements. Our
system ranked first in the 2023's DCASE Challenge, and it outperforms the
current state of the art on the ClothoV2 benchmark by 5.6 pp. mAP@10.

该研究针对预先训练的文本和频谱图变换器，提出了一种基于文本的音频检索系统。系统的两个关键组件是基于自注意力机制的音频编码器和在预训练期间利用附加的人工生成和合成数据集。该系统在 2023 年的 DCASE 挑战中排名第一，在 ClothoV2 基准测试中的表现优于当前的最新技术，提高了 5.6 个百分点的 mAP@10。