Exploring the capabilities of Large Language Models (LLMs) in puzzle solving
unveils critical insights into their potential and challenges in artificial
intelligence, marking a significant step towards understanding their
applicability in complex reasoning tasks. This survey leverages a unique
taxonomy -- dividing puzzles into rule-based and rule-less categories -- to
critically assess LLMs through various methodologies, including prompting
techniques, neuro-symbolic approaches, and fine-tuning. Through a critical
review of relevant datasets and benchmarks, we assess LLMs' performance,
identifying significant challenges in complex puzzle scenarios. Our findings
highlight the disparity between LLM capabilities and human-like reasoning,
particularly in those requiring advanced logical inference. The survey
underscores the necessity for novel strategies and richer datasets to advance
LLMs' puzzle-solving proficiency and contribute to AI's logical reasoning and
creative problem-solving advancements.

对大型语言模型（LLMs）在解谜方面的能力进行探索，揭示了它们在人工智能中的潜力和挑战，这是对其在复杂推理任务中适用性的重要一步。通过将谜题分为基于规则和非规则两类的独特分类法，该调查通过各种方法论（包括提示技术、神经符号方法和微调）对 LLMs 进行了批判性评估。通过对相关数据集和基准的批判性回顾，我们评估了 LLMs 在复杂谜题情景中的表现，发现其在需要高级逻辑推理的领域与人类推理能力存在显著差距。该调查强调了需要新的策略和更丰富数据集来提升 LLMs 在解谜方面的熟练度，并为 AI 的逻辑推理和创造性问题解决的进展做出贡献。

利用大语言模型的推理解谜：一项调查研究

Puzzle Solving using Reasoning of Large Language Models: A Survey

We explore the ability of large language models to solve and generate puzzles
from the NPR Sunday Puzzle game show using PUZZLEQA, a dataset comprising 15
years of on-air puzzles. We evaluate four large language models using PUZZLEQA,
in both multiple choice and free response formats, and explore two prompt
engineering techniques to improve free response performance: chain-of-thought
reasoning and prompt summarization. We find that state-of-the-art large
language models can solve many PUZZLEQA puzzles: the best model, GPT-3.5,
achieves 50.2% loose accuracy. However, in our few-shot puzzle generation
experiment, we find no evidence that models can generate puzzles: GPT-3.5
generates puzzles with answers that do not conform to the generated rules.
Puzzle generation remains a challenging task for future work.

使用大型语言模型和 PUZZLEQA 数据集探索解决和生成 NPR Sunday Puzzle 游戏节目谜题的能力，并发现大型语言模型可以解决 PUZZLEQA 谜题，但在生成谜题方面表现不佳，需要未来的研究。