Deception and persuasion play a critical role in long-horizon dialogues
between multiple parties, especially when the interests, goals, and motivations
of the participants are not aligned. Such complex tasks pose challenges for
current Large Language Models (LLM) as deception and persuasion can easily
mislead them, especially in long-horizon multi-party dialogues. To this end, we
explore the game of Avalon: The Resistance, a social deduction game in which
players must determine each other's hidden identities to complete their team's
objective. We introduce an online testbed and a dataset containing 20 carefully
collected and labeled games among human players that exhibit long-horizon
deception in a cooperative-competitive setting. We discuss the capabilities of
LLMs to utilize deceptive long-horizon conversations between six human players
to determine each player's goal and motivation. Particularly, we discuss the
multimodal integration of the chat between the players and the game's state
that grounds the conversation, providing further insights into the true player
identities. We find that even current state-of-the-art LLMs do not reach human
performance, making our dataset a compelling benchmark to investigate the
decision-making and language-processing capabilities of LLMs. Our dataset and
online testbed can be found at our project website:
this https URL

利用社交推理游戏 Avalon: The Resistance 中的欺骗行为和多方对话测试大型语言模型（LLMs）的能力和性能。