We introduce meta-prompting, an effective scaffolding technique designed to
enhance the functionality of language models (LMs). This approach transforms a
single LM into a multi-faceted conductor, adept at managing and integrating
multiple independent LM queries. By employing high-level instructions,
meta-prompting guides the LM to break down complex tasks into smaller, more
manageable subtasks. These subtasks are then handled by distinct "expert"
instances of the same LM, each operating under specific, tailored instructions.
Central to this process is the LM itself, in its role as the conductor, which
ensures seamless communication and effective integration of the outputs from
these expert models. It additionally employs its inherent critical thinking and
robust verification processes to refine and authenticate the end result. This
collaborative prompting approach empowers a single LM to simultaneously act as
a comprehensive orchestrator and a panel of diverse experts, significantly
enhancing its performance across a wide array of tasks. The zero-shot,
task-agnostic nature of meta-prompting greatly simplifies user interaction by
obviating the need for detailed, task-specific instructions. Furthermore, our
research demonstrates the seamless integration of external tools, such as a
Python interpreter, into the meta-prompting framework, thereby broadening its
applicability and utility. Through rigorous experimentation with GPT-4, we
establish the superiority of meta-prompting over conventional scaffolding
methods: When averaged across all tasks, including the Game of 24,
Checkmate-in-One, and Python Programming Puzzles, meta-prompting, augmented
with a Python interpreter functionality, surpasses standard prompting by 17.1%,
expert (dynamic) prompting by 17.3%, and multipersona prompting by 15.2%.

通过引入元提示技术，将单个语言模型转变为多面手指挥者，通过高级指示将复杂任务分解为可管理的子任务，从而提高语言模型的综合性能。元提示方法的任务无关性大大简化了用户交互，而与 Python 解释器等外部工具的无缝集成扩展了其适用性和实用性。在 GPT-4 上进行的实验表明，相比传统的支架方法，使用元提示技术在所有任务上平均效果提高了 17.1%，动态支架方法提高了 17.3%，多身份支架方法提高了 15.2%。

元提示：用任务无关的支架增强语言模型

Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding

As machine learning black boxes are increasingly being deployed in domains
such as healthcare and criminal justice, there is growing emphasis on building
tools and techniques for explaining these black boxes in an interpretable
manner. Such explanations are being leveraged by domain experts to diagnose
systematic errors and underlying biases of black boxes. In this paper, we
demonstrate that post hoc explanations techniques that rely on input
perturbations, such as LIME and SHAP, are not reliable. Specifically, we
propose a novel scaffolding technique that effectively hides the biases of any
given classifier by allowing an adversarial entity to craft an arbitrary
desired explanation. Our approach can be used to scaffold any biased classifier
in such a way that its predictions on the input data distribution still remain
biased, but the post hoc explanations of the scaffolded classifier look
innocuous. Using extensive evaluation with multiple real-world datasets
(including COMPAS), we demonstrate how extremely biased (racist) classifiers
crafted by our framework can easily fool popular explanation techniques such as
LIME and SHAP into generating innocuous explanations which do not reflect the
underlying biases.

本文提出一种新型的脚手架技术可以隐藏任何给定分类器的偏见，并演示了该技术如何可以欺骗常见的解释技术如 LIME 和 SHAP 生成无害的解释而不反映潜在的偏见。