Text generation is of great importance to many natural language processing
applications. However, maximization-based decoding methods (e.g. beam search)
of neural language models often lead to degenerate solutions -- the generated
text is unnatural and contains undesirable repetitions. Existing approaches
introduce stochasticity via sampling or modify training objectives to decrease
probabilities of certain tokens (e.g., unlikelihood training). However, they
often lead to solutions that lack coherence. In this work, we show that an
underlying reason for model degeneration is the anisotropic distribution of
token representations. We present a contrastive solution: (i) SimCTG, a
contrastive training objective to calibrate the model's representation space,
and (ii) a decoding method -- contrastive search -- to encourage diversity
while maintaining coherence in the generated text. Extensive experiments and
analyses on three benchmarks from two languages demonstrate that our proposed
approach significantly outperforms current state-of-the-art text generation
methods as evaluated by both human and automatic metrics.

提出了一个新的基于对比学习的方法 ——SimCTG，和一个解码方法 —— 对比搜索，用于提高文本生成质量。在两种语言的三个基准测试上，我们的方法在人类和自动评估指标方面均显著优于当前最先进的文本生成方法。