Recently, pretrained language models (PLMs) have had exceptional success in
language generation. To leverage the rich knowledge encoded by PLMs, a simple
yet powerful paradigm is to use prompts in the form of either discrete tokens
or continuous embeddings. In existing studies, these prompting methods are
typically independent of the inputs, lacking sufficient consideration of input
semantics. To address this issue, we propose a novel continuous prompting
approach, called context-tuning, to fine-tuning PLMs for natural language
generation. Firstly, the prompts are derived based on the input text to elicit
useful knowledge from PLMs for generation. We refer to such prompts as
contextualized prompts. Secondly, we use continuous inverse prompting to
improve the process of natural language generation by modeling an inverse
generation process from output to input, making the generated text more
relevant to the inputs. Furthermore, we utilize a lightweight context-tuning
method that fine-tunes only 0.12% of the parameters while maintaining good
performance. Our code is publicly available at
this https URL.

本文提出了一种新的连续提示方法，称为上下文调整，用于对预训练语言模型进行微调以进行自然语言生成，可以根据输入文本生成上下文化的提示，然后使用连续的反向提示来改进过程自然语言生成的过程。