Recent language generative models are mostly trained on large-scale datasets,
while in some real scenarios, the training datasets are often expensive to
obtain and would be small-scale. In this paper we investigate the challenging
task of less-data constrained generation, especially when the generated news
headlines are short yet expected by readers to keep readable and informative
simultaneously. We highlight the key information modeling task and propose a
novel duality fine-tuning method by formally defining the probabilistic duality
constraints between key information prediction and headline generation tasks.
The proposed method can capture more information from limited data, build
connections between separate tasks, and is suitable for less-data constrained
generation tasks. Furthermore, the method can leverage various pre-trained
generative regimes, e.g., autoregressive and encoder-decoder models. We conduct
extensive experiments to demonstrate that our method is effective and efficient
to achieve improved performance in terms of language modeling metric and
informativeness correctness metric on two public datasets.

该论文研究了有限数据情况下的标题生成任务，提出了一种新的二元优化方法来发现关键信息和生成标题任务之间的概率对偶约束。在两个公共数据集上的实验结果表明，此方法可以有效地提高语言建模指标和信息准确性指标。