BriefGPT.xyz
Jan, 2020
半自回归训练改善掩码预测解码
Semi-Autoregressive Training Improves Mask-Predict Decoding
HTML
PDF
Marjan Ghazvininejad, Omer Levy, Luke Zettlemoyer
TL;DR
该研究提出了一种新的训练方法SMART，通过模仿mask-predict的半自回归行为，使得训练样本包含模型预测作为输入，以进一步提高使用mask-predict解码的翻译质量，有效缩小了半自回归和全自回归模型之间的性能差距。
Abstract
The recently proposed
mask-predict decoding
algorithm has narrowed the performance gap between
semi-autoregressive machine translation models
and the traditional left-to-right approach. We introduce a new trainin
→