Humans can reason compositionally when presented with new tasks. Previous
research shows that appropriate prompting techniques enable large language
models (LLMs) to solve artificial compositional generalization tasks such as
SCAN. In this work, we identify additional challenges in more realistic
semantic parsing tasks with larger vocabulary and refine these prompting
techniques to address them. Our best method is based on least-to-most
prompting: it decomposes the problem using prompting-based syntactic parsing,
then uses this decomposition to select appropriate exemplars and to
sequentially generate the semantic parse. This method allows us to set a new
state of the art for CFQ while requiring only 1% of the training data used by
traditional approaches. Due to the general nature of our approach, we expect
similar efforts will lead to new results in other tasks and domains, especially
for knowledge-intensive applications.

通过逐步提示的基于句法分析的分解，本研究发现了更多语义分析任务中的挑战。这一最优方法使我们在 CFQ 上取得了最新的技术成果，同时只需要传统方法所需训练数据的 1％。由于我们方法的普适性，我们期望类似的方法将在知识密集型应用中取得新的成果。

大型语言模型的组合语义解析

Compositional Semantic Parsing with Large Language Models

Although chain-of-thought prompting has shown impressive results on many
natural language reasoning tasks, it often performs poorly on tasks which need
to solve problems harder than the demonstration examples. To tackle such
easy-to-hard generalization issues, we propose a novel prompting strategy,
least-to-most prompting. It reduces a complex problem into a list of
subproblems, and then sequentially solve these subproblems, whereby solving a
given subproblem is facilitated by the answers to previously solved
subproblems. Experiments on symbolic manipulation, compositional generalization
and math reasoning show that least-to-most prompting can generalize to the
examples that are harder than those seen in the prompt, and outperform
chain-of-thought prompting by a large margin. A notable result is that the
GPT-3 code-davinci-002 model with least-to-most-prompting solves the SCAN
benchmark regardless of splits (such as length split) with an accuracy of 99.7%
using 14 examples versus an accuracy of 16.2% by chain-of-thought prompting,
and neural-symbolic models in the literature specialized for solving SCAN are
trained with the full training set of more than 15,000 examples.

本文提出了一种新的提示策略 least-to-most prompting，将复杂问题分解为多个子问题并顺序解决，可以解决比示范范例更难的例子，并在符号操作、组合泛化和数学推理方面表现出色。