In this paper, we introduce the Tree-of-Thought (ToT) framework, a novel
approach aimed at improving the problem-solving capabilities of auto-regressive
large language models (LLMs). The ToT technique is inspired by the human mind's
approach for solving complex reasoning tasks through trial and error. In this
process, the human mind explores the solution space through a tree-like thought
process, allowing for backtracking when necessary. To implement ToT as a
software system, we augment an LLM with additional modules including a prompter
agent, a checker module, a memory module, and a ToT controller. In order to
solve a given problem, these modules engage in a multi-round conversation with
the LLM. The memory module records the conversation and state history of the
problem solving process, which allows the system to backtrack to the previous
steps of the thought-process and explore other directions from there. To verify
the effectiveness of the proposed technique, we implemented a ToT-based solver
for the Sudoku Puzzle. Experimental results show that the ToT framework can
significantly increase the success rate of Sudoku puzzle solving. Our
implementation of the ToT-based Sudoku solver is available on GitHub:
https://github.com/jieyilong/tree-of-thought-puzzle-solver.

本文提出了 Tree-of-Thought (ToT) 框架，它通过试错的思路来改进自回归大语言模型的问题解决能力。通过增加提示、检查、记忆和控制等模块来实现 ToT 系统。为了验证该技术的有效性，我们基于 ToT 设计了解决数独难题的求解器，实验结果表明，ToT 框架能够显著提高数独难题的成功率。

大型语言模型指导下的思维树

Large Language Model Guided Tree-of-Thought

Some actions must be executed in different ways depending on the context. For
example, wiping away marker requires vigorous force while wiping away almonds
requires more gentle force. In this paper we provide a model where an agent
learns which manner of action execution to use in which context, drawing on
evidence from trial and error and verbal corrections when it makes a mistake
(e.g., ``no, gently''). The learner starts out with a domain model that lacks
the concepts denoted by the words in the teacher's feedback; both the words
describing the context (e.g., marker) and the adverbs like ``gently''. We show
that through the the semantics of coherence, our agent can perform the symbol
grounding that's necessary for exploiting the teacher's feedback so as to solve
its domain-level planning problem: to perform its actions in the current
context in the right way.

该论文介绍了一种代理学习模型，在不同的上下文中学习行动执行方式，并利用试错和口头纠正来解决其领域级计划问题。

从局部纠正中学习操作方式

Learning Manner of Execution from Partial Corrections

As robots leave the controlled environments of factories to autonomously
function in more complex, natural environments, they will have to respond to
the inevitable fact that they will become damaged. However, while animals can
quickly adapt to a wide variety of injuries, current robots cannot "think
outside the box" to find a compensatory behavior when damaged: they are limited
to their pre-specified self-sensing abilities, can diagnose only anticipated
failure modes, and require a pre-programmed contingency plan for every type of
potential damage, an impracticality for complex robots. Here we introduce an
intelligent trial and error algorithm that allows robots to adapt to damage in
less than two minutes, without requiring self-diagnosis or pre-specified
contingency plans. Before deployment, a robot exploits a novel algorithm to
create a detailed map of the space of high-performing behaviors: This map
represents the robot's intuitions about what behaviors it can perform and their
value. If the robot is damaged, it uses these intuitions to guide a
trial-and-error learning algorithm that conducts intelligent experiments to
rapidly discover a compensatory behavior that works in spite of the damage.
Experiments reveal successful adaptations for a legged robot injured in five
different ways, including damaged, broken, and missing legs, and for a robotic
arm with joints broken in 14 different ways. This new technique will enable
more robust, effective, autonomous robots, and suggests principles that animals
may use to adapt to injury.

介绍了一种智能的试错算法，使机器人能够在受损后在不到两分钟内适应受损，并没有要求自诊断或预先指定应急计划。该算法可用于治疗五种不同方式的受损以及 14 种不同关节损伤的机械臂，能够让机器人更加稳健、高效和自主，同时也启示了使用自我适应原则的动物。