Multimodal question answering tasks can be used as proxy tasks to study
systems that can perceive and reason about the world. Answering questions about
different types of input modalities stresses different aspects of reasoning
such as visual reasoning, reading comprehension, story understanding, or
navigation. In this paper, we use the task of Audio Question Answering (AQA) to
study the temporal reasoning abilities of machine learning models. To this end,
we introduce the Diagnostic Audio Question Answering (DAQA) dataset comprising
audio sequences of natural sound events and programmatically generated
questions and answers that probe various aspects of temporal reasoning. We
adapt several recent state-of-the-art methods for visual question answering to
the AQA task, and use DAQA to demonstrate that they perform poorly on questions
that require in-depth temporal reasoning. Finally, we propose a new model,
Multiple Auxiliary Controllers for Linear Modulation (MALiMo) that extends the
recent Feature-wise Linear Modulation (FiLM) model and significantly improves
its temporal reasoning capabilities. We envisage DAQA to foster research on AQA
and temporal reasoning and MALiMo a step towards models for AQA.

本研究使用音频问题回答（Audio Question Answering，AQA）任务研究机器学习模型的时间推理能力，并提出 Multiple Auxiliary Controllers for Linear Modulation（MALiMo），一种扩展的 Feature-wise Linear Modulation（FiLM）模型，显著提高了其时间推理能力。