Large Language Models have shown exceptional generative abilities in various
natural language and generation tasks. However, possible anthropomorphization
and leniency towards failure cases have propelled discussions on emergent
abilities of Large Language Models especially on Theory of Mind (ToM) abilities
in Large Language Models. While several false-belief tests exists to verify the
ability to infer and maintain mental models of another entity, we study a
special application of ToM abilities that has higher stakes and possibly
irreversible consequences : Human Robot Interaction. In this work, we explore
the task of Perceived Behavior Recognition, where a robot employs a Large
Language Model (LLM) to assess the robot's generated behavior in a manner
similar to human observer. We focus on four behavior types, namely -
explicable, legible, predictable, and obfuscatory behavior which have been
extensively used to synthesize interpretable robot behaviors. The LLMs goal is,
therefore to be a human proxy to the agent, and to answer how a certain agent
behavior would be perceived by the human in the loop, for example "Given a
robot's behavior X, would the human observer find it explicable?". We conduct a
human subject study to verify that the users are able to correctly answer such
a question in the curated situations (robot setting and plan) across five
domains. A first analysis of the belief test yields extremely positive results
inflating ones expectations of LLMs possessing ToM abilities. We then propose
and perform a suite of perturbation tests which breaks this illusion, i.e.
Inconsistent Belief, Uninformative Context and Conviction Test. We conclude
that, the high score of LLMs on vanilla prompts showcases its potential use in
HRI settings, however to possess ToM demands invariance to trivial or
irrelevant perturbations in the context which LLMs lack.

通过研究大型语言模型在人机交互中的应用，本文探讨了理解机器生成行为的能力，特别是在承认他人心理状态方面，发现大型语言模型缺乏对无关紧要或微小变化的不变性。