The last couple of years have witnessed emerging research that appropriates Theory-of-Mind (ToM) tasks designed for humans to benchmark LLM's ToM capabilities as an indication of LLM's social intelligence. However, this approach has a number of limitations. Drawing on existing psychology and AI literature, we summarize the theoretical, methodological, and evaluation limitations by pointing out that certain issues are inherently present in the original ToM tasks used to evaluate human's ToM, which continues to persist and exacerbated when appropriated to benchmark LLM's ToM. Taking a human-computer interaction (HCI) perspective, these limitations prompt us to rethink the definition and criteria of ToM in ToM benchmarks in a more dynamic, interactional approach that accounts for user preferences, needs, and experiences with LLMs in such evaluations. We conclude by outlining potential opportunities and challenges towards this direction.

本研究针对现有的基于人类心智理论任务评估大型语言模型（LLMs）社交智能的局限性进行了探讨。我们提出了一种人机交互视角的动态和互动的方法，重新定义和修订了心智理论基准，以更好地反映用户的偏好、需求和体验。这一方法的主要发现是，改进的基准可以提高对LLM心智理论能力的评估准确性和实用性。

重新思考面向用户的LLM心智理论基准