The issue of hallucinations is a prevalent concern in existing Large
Vision-Language Models (LVLMs). Previous efforts have primarily focused on
investigating object hallucinations, which can be easily alleviated by
introducing object detectors. However, these efforts neglect hallucinations in
inter-object relationships, which is essential for visual comprehension. In
this work, we introduce R-Bench, a novel benchmark for evaluating Vision
Relationship Hallucination. R-Bench features image-level questions that focus
on the existence of relationships and instance-level questions that assess
local visual comprehension. We identify three types of relationship
co-occurrences that lead to hallucinations: relationship-relationship,
subject-relationship, and relationship-object. The visual instruction tuning
dataset's long-tail distribution significantly impacts LVLMs' understanding of
visual relationships. Furthermore, our analysis reveals that current LVLMs tend
to disregard visual content and overly rely on the common sense knowledge of
Large Language Models. They also struggle with reasoning about spatial
relationships based on contextual information.

通过引入 R-Bench 评估 Vision Relationship Hallucination，在大型视觉语言模型中探讨幻觉问题，尤其关注对象关系中的幻觉，包括关系 - 关系、主体 - 关系和关系 - 对象三种关系共现形成的幻觉，以及视觉内容和基于上下文的空间关系推理的困难。