Performance of large language models (LLMs) may vary with different prompts
or instructions of even the same task. One commonly recognized factor for this
phenomenon is the model's familiarity with the given prompt or instruction,
which is typically estimated by its perplexity. However, finding the prompt
with the lowest perplexity is challenging, given the enormous space of possible
prompting phrases. In this paper, we propose monotonic paraphrasing (MonoPara),
an end-to-end decoding strategy that paraphrases given prompts or instructions
into their lower perplexity counterparts based on an ensemble of a paraphrase
LM for prompt (or instruction) rewriting, and a target LM (i.e. the prompt or
instruction executor) that constrains the generation for lower perplexity. The
ensemble decoding process can efficiently paraphrase the original prompt
without altering its semantic meaning, while monotonically decreasing the
perplexity of each generation as calculated by the target LM. We explore in
detail both greedy and search-based decoding as two alternative decoding
schemes of MonoPara. Notably, MonoPara does not require any training and can
monotonically lower the perplexity of the paraphrased prompt or instruction,
leading to improved performance of zero-shot LM prompting as evaluated on a
wide selection of tasks. In addition, MonoPara is also shown to effectively
improve LMs' generalization on perturbed and unseen task instructions.

我们提出了一种单调改写方法（MonoPara），通过一个将提示或指令改写成更低困惑度的改写语言模型和一个用于限制生成更低困惑度的目标语言模型的组合解码过程，对原始提示进行改写而不改变其语义含义，从而降低生成的困惑度，提高零样本语言模型的表现，并且有效改善语言模型对扰动和未见任务指令的泛化能力。

单调改写提高语言模型启示的泛化能力

Monotonic Paraphrasing Improves Generalization of Language Model  Prompting

Language model (LM) prompting--a popular paradigm for solving NLP tasks--has
been shown to be susceptible to miscalibration and brittleness to slight prompt
variations, caused by its discriminative prompting approach, i.e., predicting
the label given the input. To address these issues, we propose Gen-Z--a
generative prompting framework for zero-shot text classification. GEN-Z is
generative, as it measures the LM likelihood of input text, conditioned on
natural language descriptions of labels. The framework is multivariate, as
label descriptions allow us to seamlessly integrate additional contextual
information about the labels to improve task performance. On various standard
classification benchmarks, with six open-source LM families, we show that
zero-shot classification with simple contextualization of the data source of
the evaluation set consistently outperforms both zero-shot and few-shot
baselines while improving robustness to prompt variations. Further, our
approach enables personalizing classification in a zero-shot manner by
incorporating author, subject, or reader information in the label descriptions.

Gen-Z 是一种用于零食文本分类的生成提示框架，通过在标签的自然语言描述上依赖语言模型的可能性来改善任务性能，并在多个标准分类基准上展现了优越性能。