Large audio-language models (LALMs) enhance traditional large language models
by integrating audio perception capabilities, allowing them to tackle
audio-related tasks. Previous research has primarily focused on assessing the
performance of LALMs across various tasks, yet overlooking their reliability,
particularly concerning issues like object hallucination. In our study, we
introduce methods to assess the extent of object hallucination of publicly
available LALMs. Our findings reveal that LALMs are comparable to specialized
audio captioning models in their understanding of audio content, but struggle
to answer discriminative questions, specifically those requiring the
identification of the presence of particular object sounds within an audio
clip. This limitation highlights a critical weakness in current LALMs: their
inadequate understanding of discriminative queries. Moreover, we explore the
potential of prompt engineering to enhance LALMs' performance on discriminative
questions.

现有的大型音频语言模型在集成音频感知能力方面扩展了传统大型语言模型的功能，对其性能在各种任务上进行了主要的评估，但忽视了它们的可靠性，特别是与对象幻觉等问题相关的可靠性。本研究引入了评估公开可用的大型音频语言模型的对象幻觉程度的方法。研究发现，大型音频语言模型在理解音频内容方面与专用音频字幕模型相媲美，但在回答鉴别性问题上遇到困难，尤其是要求识别音频剪辑中特定对象声音存在的问题。这个限制凸显了当前大型音频语言模型的一个关键弱点：对于鉴别性查询的理解不足。此外，我们还探索了提示工程在提高大型音频语言模型在鉴别性问题上的性能方面的潜力。