Pre-trained Transformers have enabled impressive breakthroughs in generating
long and fluent text, yet their outputs are often "rambling" without coherently
arranged content. In this work, we present a novel content-controlled text
generation framework, PAIR, with planning and iterative refinement, which is
built upon a large model, BART. We first adapt the BERT model to automatically
construct the content plans, consisting of keyphrase assignments and their
corresponding sentence-level positions. The BART model is employed for
generation without modifying its structure. We then propose a refinement
algorithm to gradually enhance the generation quality within the
sequence-to-sequence framework. Evaluation with automatic metrics shows that
adding planning consistently improves the generation quality on three distinct
domains, with an average of 20 BLEU points and 12 METEOR points improvements.
In addition, human judges rate our system outputs to be more relevant and
coherent than comparisons without planning.

本文介绍了一种基于预训练转换器的内容控制型文本生成框架 ——PAIR。该框架采用了规划和迭代改进策略，并通过 BART 模型实现了生成。采用自动评估方法得分表明，添加规划可显著提高文本生成的质量，同时人工评估结果表明，PAIR 系统的生成结果相对于没有规划的对比组更加相关和连贯。