Large Language Models (LLMs) have recently shown a promise and emergence of
Theory of Mind (ToM) ability and even outperform humans in certain ToM tasks.
To evaluate and extend the boundaries of the ToM reasoning ability of LLMs, we
propose a novel concept, taxonomy, and framework, the ToM reasoning with Zero,
Finite, and Infinite Belief History and develop a multi-round text-based game,
called $\textit{Pick the Right Stuff}$, as a benchmark. We have evaluated six
LLMs with this game and found their performance on Zero Belief History is
consistently better than on Finite Belief History. In addition, we have found
two of the models with small parameter sizes outperform all the evaluated
models with large parameter sizes. We expect this work to pave the way for
future ToM benchmark development and also for the promotion and development of
more complex AI agents or systems which are required to be equipped with more
complex ToM reasoning ability.

在这篇论文中，我们提出了一种新的概念、分类法和框架 —— 使用零、有限和无限信仰历史进行理论心智推理，并开发了一个名为 'Pick the Right Stuff' 的多回合文本游戏作为一个基准。我们评估了六个大型语言模型在这个游戏中的表现，并发现它们在零信仰历史上的性能一致优于有限信仰历史。此外，我们还发现其中两个参数规模较小的模型胜过了所有参数规模较大的模型。我们期望这项工作为未来的理论心智基准开发铺平道路，也为需要更复杂的理论心智推理能力的更复杂的人工智能代理或系统的推进和发展提供支持。