With the rapid development of LLMs, it is natural to ask how to harness their
capabilities efficiently. In this paper, we explore whether it is feasible to
direct each input query to a single most suitable LLM. To this end, we propose
LLM routing for challenging reasoning tasks. Our extensive experiments suggest
that such routing shows promise but is not feasible in all scenarios, so more
robust approaches should be investigated to fill this gap.

我们探讨了是否可以将每个输入查询直接路由到最合适的 LLM，我们的广泛实验表明这样的路由有前景但在所有情景下都不可行，因此应该研究更加稳健的方法来填补这一空白。

发挥多重智慧的力量：从 LLM 路由中所学到的经验

Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routing

Large Language Models (LLMs) have demonstrated impressive planning abilities
due to their vast "world knowledge". Yet, obtaining plans that are both
feasible (grounded in affordances) and cost-effective (in plan length), remains
a challenge, despite recent progress. This contrasts with heuristic planning
methods that employ domain knowledge (formalized in action models such as PDDL)
and heuristic search to generate feasible, optimal plans. Inspired by this, we
propose to combine the power of LLMs and heuristic planning by leveraging the
world knowledge of LLMs and the principles of heuristic search. Our approach,
SayCanPay, employs LLMs to generate actions (Say) guided by learnable domain
knowledge, that evaluates actions' feasibility (Can) and long-term
reward/payoff (Pay), and heuristic search to select the best sequence of
actions. Our contributions are (1) a novel framing of the LLM planning problem
in the context of heuristic planning, (2) integrating grounding and
cost-effective elements into the generated plans, and (3) using heuristic
search over actions. Our extensive evaluations show that our model surpasses
other LLM planning approaches.

结合大型语言模型和启发式规划的方法，该研究提出了一种通过利用大型语言模型的世界知识和启发式搜索原则来生成行动序列的新方法，该方法能够在规划过程中考虑行动的可行性和成本效益，评估行动的长期效果，并在广泛的评估中显示优于其他大型语言模型规划方法。

SayCanPay: 基于可学习领域知识的大型语言模型的启发式规划

SayCanPay: Heuristic Planning with Large Language Models using Learnable  Domain Knowledge

Calls to arms to build interpretable models express a well-founded discomfort
with machine learning. Should a software agent that does not even know what a
loan is decide who qualifies for one? Indeed, we ought to be cautious about
injecting machine learning (or anything else, for that matter) into
applications where there may be a significant risk of causing social harm.
However, claims that stakeholders "just won't accept that!" do not provide a
sufficient foundation for a proposed field of study. For the field of
interpretable machine learning to advance, we must ask the following questions:
What precisely won't various stakeholders accept? What do they want? Are these
desiderata reasonable? Are they feasible? In order to answer these questions,
we'll have to give real-world problems and their respective stakeholders
greater consideration.

机器学习的不可解释性给人带来了很大的困扰，因此需要建立可解释模型，以规避应用中可能引起社会伤害的风险。为了推进机器学习的可解释性，我们需要重视现实问题及其相关利益相关者的要求，将其可行性进行考虑。