To guide the generation of large pretrained language models (LM), previous
work has focused on directly fine-tuning the language model or utilizing an
attribute discriminator. In this work, we propose a novel lightweight framework
for controllable GPT2 generation, which utilizes a set of small
attribute-specific vectors, called prefixes, to steer natural language
generation. Different from prefix-tuning, where each prefix is trained
independently, we take the relationship among prefixes into consideration and
train multiple prefixes simultaneously. We propose a novel supervised method
and also an unsupervised method to train the prefixes for single-aspect control
while the combination of these two methods can achieve multi-aspect control.
Experimental results on both single-aspect and multi-aspect control show that
our methods can guide generation towards the desired attributes while keeping
high linguistic quality.

提出了一种新颖的轻量级的 GPT2 生成框架，该框架利用一组小的属性特定向量，称为前缀，来控制自然语言生成，其中前缀使用多方面控制的监督方法和无监督方法进行训练，实验结果表明，我们的方法可以引导生成达到期望的特性，同时保持高水平的语言质量。