Given a task in the form of a basic description and its training examples,
prompt optimization is the problem of synthesizing the given information into a
text prompt for a large language model (LLM). Humans solve this problem by also
considering the different facets that define a task (e.g., counter-examples,
explanations, analogies) and including them in the prompt. However, it is
unclear whether existing algorithmic approaches, based on iteratively editing a
given prompt or automatically selecting a few in-context examples, can cover
the multiple facets required to solve a complex task. In this work, we view
prompt optimization as that of learning multiple facets of a task from a set of
training examples. We identify and exploit structure in the prompt optimization
problem -- first, we find that prompts can be broken down into loosely coupled
semantic sections that have a relatively independent effect on the prompt's
performance; second, we cluster the input space and use clustered batches so
that the optimization procedure can learn the different facets of a task across
batches. The resulting algorithm, UniPrompt, consists of a generative model to
generate initial candidates for each prompt section; and a feedback mechanism
that aggregates suggested edits from multiple mini-batches into a conceptual
description for the section. Empirical evaluation on multiple datasets and a
real-world task shows that prompts generated using UniPrompt obtain higher
accuracy than human-tuned prompts and those from state-of-the-art methods. In
particular, our algorithm can generate long, complex prompts that existing
methods are unable to generate. Code for UniPrompt will be available at
https://aka.ms/uniprompt.

根据给定的训练样本和基本描述，我们将提示优化问题视为从训练样本集中学习任务的多个方面，并发现了提示优化问题中的结构，提出了 UniPrompt 算法，通过生成模型生成初始候选提示部分，并使用反馈机制将多个小批次的建议编辑聚合成一个概念描述，实验证明 UniPrompt 生成的提示比人工调优的提示和最先进方法的提示具有更高的准确性。

任务特征学习：一种用于提示优化的结构化方法

Task Facet Learning: A Structured Approach to Prompt Optimization

Prompt-based tuning has been proven effective for pretrained language models
(PLMs). While most of the existing work focuses on the monolingual prompts, we
study the multilingual prompts for multilingual PLMs, especially in the
zero-shot cross-lingual setting. To alleviate the effort of designing different
prompts for multiple languages, we propose a novel model that uses a unified
prompt for all languages, called UniPrompt. Different from the discrete prompts
and soft prompts, the unified prompt is model-based and language-agnostic.
Specifically, the unified prompt is initialized by a multilingual PLM to
produce language-independent representation, after which is fused with the text
input. During inference, the prompts can be pre-computed so that no extra
computation cost is needed. To collocate with the unified prompt, we propose a
new initialization method for the target label word to further improve the
model's transferability across languages. Extensive experiments show that our
proposed methods can significantly outperform the strong baselines across
different languages. We release data and code to facilitate future research.

本文提出了一种使用 UniPrompt 的新型模型，其使用单一的多语言 prompt 并且具有语言不可知性，可以显著提高在零样本跨语言设置下的转移性能。