Despite the significant advancements in natural language processing
capabilities demonstrated by large language models such as ChatGPT, their
proficiency in comprehending and processing spatial information, especially
within the domains of 2D and 3D route planning, remains notably underdeveloped.
This paper investigates the inherent limitations of ChatGPT and similar models
in spatial reasoning and navigation-related tasks, an area critical for
applications ranging from autonomous vehicle guidance to assistive technologies
for the visually impaired. In this paper, we introduce a novel evaluation
framework complemented by a baseline dataset, meticulously crafted for this
study. This dataset is structured around three key tasks: plotting spatial
points, planning routes in two-dimensional (2D) spaces, and devising pathways
in three-dimensional (3D) environments. We specifically developed this dataset
to assess the spatial reasoning abilities of ChatGPT. Our evaluation reveals
key insights into the model's capabilities and limitations in spatial
understanding.

该论文研究了 ChatGPT 和类似模型在空间推理和导航相关任务中的固有限制，并引入了一种新颖的评估框架和基准数据集以评估 ChatGPT 的空间推理能力。研究揭示了模型在空间理解方面的能力和局限性的关键见解。