Prompt engineering is a challenging yet crucial task for optimizing the
performance of large language models (LLMs). It requires complex reasoning to
examine the model's errors, hypothesize what is missing or misleading in the
current prompt, and communicate the task with clarity. While recent works
indicate that LLMs can be meta-prompted to perform automatic prompt
engineering, their potentials may not be fully untapped due to the lack of
sufficient guidance to elicit complex reasoning capabilities in LLMs in the
meta-prompt. In this work, we investigate the problem of "prompt engineering a
prompt engineer" -- constructing a meta-prompt that more effectively guides
LLMs to perform automatic prompt engineering. We introduce and analyze key
components, such as a step-by-step reasoning template and context
specification, which lead to improved performance. In addition, inspired by
common optimization concepts such as batch size, step size and momentum, we
introduce their verbalized counterparts to the meta-prompt and investigate
their effects. Our final method, named PE2, finds a prompt that outperforms
"let's think step by step" by 6.3% on the MultiArith dataset and 3.1% on the
GSM8K dataset. To demonstrate its versatility, we apply PE2 to the Instruction
Induction benchmark, a suite of counterfactual tasks, and a lengthy, real-world
industrial prompt. In these settings, PE2 achieves strong performance and
outperforms prior automatic prompt engineering baselines. Further, we show that
PE2 makes meaningful and targeted prompt edits, amends erroneous or incomplete
prompts, and presents non-trivial counterfactual reasoning abilities.

通过构建元提示（meta-prompt）从而更有效地引导大型语言模型（LLMs）进行自动提示工程，进而优化其性能。

提示工程师的提示工程

Prompt Engineering a Prompt Engineer

We present Diversity-Aware Meta Visual Prompting~(DAM-VP), an efficient and
effective prompting method for transferring pre-trained models to downstream
tasks with frozen backbone. A challenging issue in visual prompting is that
image datasets sometimes have a large data diversity whereas a per-dataset
generic prompt can hardly handle the complex distribution shift toward the
original pretraining data distribution properly. To address this issue, we
propose a dataset Diversity-Aware prompting strategy whose initialization is
realized by a Meta-prompt. Specifically, we cluster the downstream dataset into
small homogeneity subsets in a diversity-adaptive way, with each subset has its
own prompt optimized separately. Such a divide-and-conquer design reduces the
optimization difficulty greatly and significantly boosts the prompting
performance. Furthermore, all the prompts are initialized with a meta-prompt,
which is learned across several datasets. It is a bootstrapped paradigm, with
the key observation that the prompting knowledge learned from previous datasets
could help the prompt to converge faster and perform better on a new dataset.
During inference, we dynamically select a proper prompt for each input, based
on the feature distance between the input and each subset. Through extensive
experiments, our DAM-VP demonstrates superior efficiency and effectiveness,
clearly surpassing previous prompting methods in a series of downstream
datasets for different pretraining models. Our code is available at:
https://github.com/shikiw/DAM-VP.

本文提出了一种称为 Diversity-Aware Meta Visual Prompting 的有效提示方法，可将预训练模型传递到冻结的下游任务，旨在解决图像数据集多样性大的挑战性问题。该方法包括数据集多样性感知提示策略，集群下游数据集到具有相似性的小子集，给每个子集优化一个提示，所有提示都由一个元提示初始化，通过推断过程动态选择适当的提示。实验结果表明，DAM-VP 优于以前的提示方法。