BriefGPT.xyz
Jun, 2022
学习打破循环:分析和减少神经文本生成中的重复
Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation
HTML
PDF
Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, Huayang Li...
TL;DR
通过量化实验研究,我们发现神经语言模型生成文本中会存在连续重复句子的现象,并提出了针对该现象的训练方法DITTO,该方法不仅可以缓解生成中的重复问题,同时还能提高生成质量。
Abstract
While large-scale
neural language models
, such as GPT2 and BART, have achieved impressive results on various
text generation
tasks, they tend to get stuck in undesirable sentence-level loops with maximization-bas
→