Performance of large language models (LLMs) may vary with different prompts
or instructions of even the same task. One commonly recognized factor for this
phenomenon is the model's familiarity with the given prompt or instruction,
which is typically estimated by its perplexity. However, finding the prompt
with the lowest perplexity is challenging, given the enormous space of possible
prompting phrases. In this paper, we propose monotonic paraphrasing (MonoPara),
an end-to-end decoding strategy that paraphrases given prompts or instructions
into their lower perplexity counterparts based on an ensemble of a paraphrase
LM for prompt (or instruction) rewriting, and a target LM (i.e. the prompt or
instruction executor) that constrains the generation for lower perplexity. The
ensemble decoding process can efficiently paraphrase the original prompt
without altering its semantic meaning, while monotonically decreasing the
perplexity of each generation as calculated by the target LM. We explore in
detail both greedy and search-based decoding as two alternative decoding
schemes of MonoPara. Notably, MonoPara does not require any training and can
monotonically lower the perplexity of the paraphrased prompt or instruction,
leading to improved performance of zero-shot LM prompting as evaluated on a
wide selection of tasks. In addition, MonoPara is also shown to effectively
improve LMs' generalization on perturbed and unseen task instructions.

我们提出了一种单调改写方法（MonoPara），通过一个将提示或指令改写成更低困惑度的改写语言模型和一个用于限制生成更低困惑度的目标语言模型的组合解码过程，对原始提示进行改写而不改变其语义含义，从而降低生成的困惑度，提高零样本语言模型的表现，并且有效改善语言模型对扰动和未见任务指令的泛化能力。