Neural Theory-of-Mind (N-ToM), machine's ability to understand and keep track
of the mental states of others, is pivotal in developing socially intelligent
agents. However, prevalent N-ToM benchmarks have several shortcomings,
including the presence of ambiguous and artificial narratives, absence of
personality traits and preferences, a lack of questions addressing characters'
psychological mental states, and limited diversity in the questions posed. In
response to these issues, we construct OpenToM, a new benchmark for assessing
N-ToM with (1) longer and clearer narrative stories, (2) characters with
explicit personality traits, (3) actions that are triggered by character
intentions, and (4) questions designed to challenge LLMs' capabilities of
modeling characters' mental states of both the physical and psychological
world. Using OpenToM, we reveal that state-of-the-art LLMs thrive at modeling
certain aspects of mental states in the physical world but fall short when
tracking characters' mental states in the psychological world.

机器学习中的神经心智理论 (N-ToM) 是理解和跟踪他人心理状态的关键，本研究构建了一个新的基准 OpenToM，通过长而清晰的叙述、具有明确人格特征的角色以及挑战人工智能模型对心理和心理世界中角色心理状态建模能力的设计提出了一些问题，揭示了最先进的模型在物理世界中对心理状态的建模方面表现出色，但在心理世界中的心理状态跟踪方面表现不足。