Despite recent progress of pre-trained language models on generating fluent
text, existing methods still suffer from incoherence problems in long-form text
generation tasks that require proper content control and planning to form a
coherent high-level logical flow. In this work, we propose PLANET, a novel
generation framework leveraging autoregressive self-attention mechanism to
conduct content planning and surface realization dynamically. To guide the
generation of output sentences, our framework enriches the Transformer decoder
with latent representations to maintain sentence-level semantic plans grounded
by bag-of-words. Moreover, we introduce a new coherence-based contrastive
learning objective to further improve the coherence of output. Extensive
experiments are conducted on two challenging long-form text generation tasks
including counterargument generation and opinion article generation. Both
automatic and human evaluations show that our method significantly outperforms
strong baselines and generates more coherent texts with richer contents.

本文提出了 PLANET 框架，采用自回归自注意机制，结合包含词汇的句子级语义计划来动态地进行内容规划和表面实现。引入基于连贯性的对比学习目标，以进一步提高输出的连贯性。实验结果表明，我们的方法在生成较富内容的连贯文本方面优于基线方法。