Recent work found high mutual information between the learned representations of large language models (LLMs) and the geospatial property of its input, hinting an emergent internal model of space. However, whether this internal space model has any causal effects on the LLMs' behaviors was not answered by that work, led to criticism of these findings as mere statistical correlation. Our study focused on uncovering the causality of the spatial representations in LLMs. In particular, we discovered the potential spatial representations in DeBERTa, GPT-Neo using representational similarity analysis and linear and non-linear probing. Our casual intervention experiments showed that the spatial representations influenced the model's performance on next word prediction and a downstream task that relies on geospatial information. Our experiments suggested that the LLMs learn and use an internal model of space in solving geospatial related tasks.

通过表示相似性分析和线性和非线性探测，我们发现了DeBERTa和GPT-Neo中潜在的空间表示，并通过因果干预实验证明了空间表示对模型的下一个词预测性能和依赖地理空间信息的下游任务的影响，这些实验证明了大型语言模型在解决地理空间相关任务时学习和使用内部空间模型。

超越相关性：大型语言模型是否学习到空间的因果表征？