This paper presents an innovative framework that integrates Large Language
Models (LLMs) with an external Thinker module to enhance the reasoning
capabilities of LLM-based agents. Unlike augmenting LLMs with prompt
engineering, Thinker directly harnesses knowledge from databases and employs
various optimization techniques. The framework forms a reasoning hierarchy
where LLMs handle intuitive System-1 tasks such as natural language processing,
while the Thinker focuses on cognitive System-2 tasks that require complex
logical analysis and domain-specific knowledge. Our framework is presented
using a 9-player Werewolf game that demands dual-system reasoning. We introduce
a communication protocol between LLMs and the Thinker, and train the Thinker
using data from 18800 human sessions and reinforcement learning. Experiments
demonstrate the framework's effectiveness in deductive reasoning, speech
generation, and online game evaluation. Additionally, we fine-tune a 6B LLM to
surpass GPT4 when integrated with the Thinker. This paper also contributes the
largest dataset for social deduction games to date.

本文提出了一种创新的框架，将大型语言模型（LLMs）与外部思考器模块相结合，以增强基于 LLM 的代理机构的推理能力。该框架形成了一个推理层次结构，其中 LLMs 处理直觉性的 System-1 任务，而思考器专注于需要复杂逻辑分析和领域特定知识的认知 System-2 任务。实验证明了该框架在演绎推理、语音生成和在线游戏评估方面的有效性。此外，我们通过与思考器集成来调优 6B LLM，以超越 GPT4。本文还贡献了迄今为止最大的社交推理游戏数据集。