Matching raw audio signals with textual descriptions requires understanding
the audio's content and the description's semantics and then drawing
connections between the two modalities. This paper investigates a hybrid
retrieval system that utilizes audio metadata as an additional clue to
understand the content of audio signals before matching them with textual
queries. We experimented with metadata often attached to audio recordings, such
as keywords and natural-language descriptions, and we investigated late and
mid-level fusion strategies to merge audio and metadata. Our hybrid approach
with keyword metadata and late fusion improved the retrieval performance over a
content-based baseline by 2.36 and 3.69 pp. mAP@10 on the ClothoV2 and
AudioCaps benchmarks, respectively.

通过利用音频元数据作为额外线索，以了解音频信号内容并将其与文本查询进行匹配，本文研究了一种混合检索系统。我们通过实验使用通常附加在音频录音上的元数据，如关键词和自然语言描述，并调查了融合音频和元数据的后期和中间级融合策略。我们的混合方法通过关键词元数据和后期融合，相较于基于内容的基线，分别在 ClothoV2 和 AudioCaps 基准上将检索性能提高了 2.36 和 3.69 个点，mAP@10。