Advancements in large language models (LLMs) are revolutionizing interactive
game design, enabling dynamic plotlines and interactions between players and
non-player characters (NPCs). However, LLMs may exhibit flaws such as
hallucinations, forgetfulness, or misinterpretations of prompts, causing
logical inconsistencies and unexpected deviations from intended designs.
Automated techniques for detecting such game bugs are still lacking. To address
this, we propose a systematic LLM-based method for automatically identifying
such bugs from player game logs, eliminating the need for collecting additional
data such as post-play surveys. Applied to a text-based game DejaBoom!, our
approach effectively identifies bugs inherent in LLM-powered interactive games,
surpassing unstructured LLM-powered bug-catching methods and filling the gap in
automated detection of logical and design flaws.

通过基于大型语言模型的方法，我们提出了一种自动识别互动游戏中逻辑和设计缺陷的系统化方法，该方法应用于文本游戏 DejaBoom! 上能够高效地识别大型语言模型驱动的互动游戏中的缺陷，并填补了自动检测逻辑和设计缺陷的空白。