The Multi-Modal Large Language Model (MLLM) refers to an extension of the
Large Language Model (LLM) equipped with the capability to receive and infer
multi-modal data. Spatial awareness stands as one of the crucial abilities of
MLLM, encompassing diverse skills related to understanding spatial
relationships among objects and between objects and the scene area. Industries
such as autonomous driving, smart healthcare, robotics, virtual, and augmented
reality heavily demand MLLM's spatial awareness capabilities. However, there
exists a noticeable gap between the current spatial awareness capabilities of
MLLM and the requirements set by human needs. To address this issue, this paper
proposes using more precise spatial position information between objects to
guide MLLM in providing more accurate responses to user-related inquiries.
Specifically, for a particular multi-modal task, we utilize algorithms for
acquiring geometric spatial information and scene graphs to obtain relevant
geometric spatial information and scene details of objects involved in the
query. Subsequently, based on this information, we direct MLLM to address
spatial awareness-related queries posed by the user. Extensive experiments were
conducted in benchmarks such as MME, MM-Vet, and other multi-modal large
language models. The experimental results thoroughly confirm the efficacy of
the proposed method in enhancing the spatial awareness tasks and associated
tasks of MLLM.

本文针对多模态大型语言模型（MLLM）在空间感知能力与人类需求之间存在的差距，提出使用更精确的物体间空间位置信息来引导 MLLM 以更准确地回答用户相关查询，并通过实验证明该方法在增强 MLLM 的空间感知任务和相关任务方面的有效性。