Large vision language models have demonstrated remarkable efficacy in
addressing challenges related to both textual and visual content. Nevertheless,
these models are susceptible to various hallucinations. In this paper, we focus
on a new form of hallucination, specifically termed as number hallucination,
which denotes instances where models fail to accurately identify the quantity
of objects in an image. We establish a dataset and employ evaluation metrics to
assess number hallucination, revealing a pronounced prevalence of this issue
across mainstream large vision language models (LVLMs). Additionally, we delve
into a thorough analysis of number hallucination, examining inner and outer
inconsistency problem from two related perspectives. We assert that this
inconsistency is one cause of number hallucination and propose a consistency
training method as a means to alleviate such hallucination, which achieves an
average improvement of 8\% compared with direct finetuning method.

通过建立数据集和使用评估指标，我们发现大规模视觉语言模型中存在数量幻觉的普遍问题，并从内部和外部一致性问题的角度进行了深入分析。我们提出了一种一致性训练方法来减轻这种幻觉，与直接微调方法相比，其平均改善了 8%。