Recently, contrastive learning attracts increasing interests in neural text
generation as a new solution to alleviate the exposure bias problem. It
introduces a sequence-level training signal which is crucial to generation
tasks that always rely on auto-regressive decoding. However, previous methods
using contrastive learning in neural text generation usually lead to inferior
performance. In this paper, we analyse the underlying reasons and propose a new
Contrastive Neural Text generation framework, CoNT. CoNT addresses bottlenecks
that prevent contrastive learning from being widely adopted in generation tasks
from three aspects -- the construction of contrastive examples, the choice of
the contrastive loss, and the strategy in decoding. We validate CoNT on five
generation tasks with ten benchmarks, including machine translation,
summarization, code comment generation, data-to-text generation and commonsense
generation. Experimental results show that CoNT clearly outperforms the
conventional training framework on all the ten benchmarks with a convincing
margin. Especially, CoNT surpasses previous the most competitive contrastive
learning method for text generation, by 1.50 BLEU on machine translation and
1.77 ROUGE-1 on summarization, respectively. It achieves new state-of-the-art
on summarization, code comment generation (without external data) and
data-to-text generation.

本文提出了一种新的对比学习框架 CoNT，它从对比示例、对比损失和解码策略三个方面解决了对比学习在生成任务中使用的瓶颈问题，实验结果表明 CoNT 在机器翻译、数据摘要、代码注释生成等十个基准测试中都表现出优异的表现，特别是在摘要生成任务上达到了新的最好结果。