Audio question answering (AQA), acting as a widely used proxy task to explore
scene understanding, has got more attention. The AQA is challenging for it
requires comprehensive temporal reasoning from different scales' events of an
audio scene. However, existing methods mostly extend the structures of visual
question answering task to audio ones in a simple pattern but may not perform
well when perceiving a fine-grained audio scene. To this end, we present a
Multi-scale Window Attention Fusion Model (MWAFM) consisting of an asynchronous
hybrid attention module and a multi-scale window attention module. The former
is designed to aggregate unimodal and cross-modal temporal contexts, while the
latter captures sound events of varying lengths and their temporal dependencies
for a more comprehensive understanding. Extensive experiments are conducted to
demonstrate that the proposed MWAFM can effectively explore temporal
information to facilitate AQA in the fine-grained scene.Code:
this https URL

本篇研究提出了一种多尺度窗口注意力融合模型，通过异步混合注意力模块和多尺度窗口注意力模块，提高了音频场景理解中的时间推理能力。