We investigate the abilities of a representative set of Large language Models
(LLMs) to reason about cardinal directions (CDs). To do so, we create two
datasets: the first, co-created with ChatGPT, focuses largely on recall of
world knowledge about CDs; the second is generated from a set of templates,
comprehensively testing an LLM's ability to determine the correct CD given a
particular scenario. The templates allow for a number of degrees of variation
such as means of locomotion of the agent involved, and whether set in the first
, second or third person. Even with a temperature setting of zero, Our
experiments show that although LLMs are able to perform well in the simpler
dataset, in the second more complex dataset no LLM is able to reliably
determine the correct CD, even with a temperature setting of zero.

探讨大型语言模型在基于基数方向的推理能力方面的研究，通过创建两个数据集，发现尽管在简单数据集中大型语言模型表现良好，但在更复杂的数据集中，即使温度设置为零，没有任何大型语言模型能够可靠地确定正确的基数方向。

评估大型语言模型在理解基数方向上的能力

Evaluating the Ability of Large Language Models to Reason about Cardinal  Directions

We propose a novel formal framework (called 3D-nCDC-ASP) to represent and
reason about cardinal directions between extended objects in 3-dimensional (3D)
space, using Answer Set Programming (ASP). 3D-nCDC-ASP extends Cardinal
Directional Calculus (CDC) with a new type of default constraints, and nCDC-ASP
to 3D. 3D-nCDC-ASP provides a flexible platform offering different types of
reasoning: Nonmonotonic reasoning with defaults, checking consistency of a set
of constraints on 3D cardinal directions between objects, explaining
inconsistencies, and inferring missing CDC relations. We prove the soundness of
3D-nCDC-ASP, and illustrate its usefulness with applications. This paper is
under consideration for acceptance in TPLP.

本文提出了一种新的形式框架（称为 3D-nCDC-ASP），用于利用 ASP 在三维（3D）空间中表示和推理延伸对象之间的基准方向。3D-nCDC-ASP 基于 CDC 和 nCDC-ASP，提供灵活的平台，为不同类型的推理提供支持，证明了其完备性，并说明其应用价值。