Neural language models are probabilistic models of human text. They are
predominantly trained using maximum likelihood estimation (MLE), which is
equivalent to minimizing the forward cross-entropy between the empirical data
distribution and the model distribution. However, various degeneration
phenomena are still widely observed when decoding from the distributions
learned by such models. We establish that the forward cross-entropy is
suboptimal as a distance metric for aligning human and model distribution due
to its (1) recall-prioritization (2) negative diversity ignorance and (3)
train-test mismatch. In this paper, we propose Earth Mover Distance
Optimization (EMO) for auto-regressive language modeling. EMO capitalizes on
the inherent properties of earth mover distance to address the aforementioned
challenges. Due to the high complexity of direct computation, we further
introduce a feasible upper bound for EMO to ease end-to-end training. Upon
extensive evaluation of language models trained using EMO and MLE. We find that
EMO demonstrates a consistently better language modeling performance than MLE
across domains. Moreover, EMO demonstrates noteworthy enhancements in
downstream performance with minimal fine-tuning on merely 25,000 sentences.
This highlights the tremendous potential of EMO as a lightweight calibration
method for enhancing large-scale pre-trained language models.

神经语言模型是人类文本的概率模型，主要使用最大似然估计进行训练。本文提出了基于 EMD 优化的自回归语言建模方法，通过对 EMD 的上界估计实现了端到端训练，并在广泛评估中表现出了比 MLE 更好的语言建模性能。此外，EMO 还能在仅微调 25000 个句子的情况下大幅提升下游任务性能，展现了作为轻量级校准方法的巨大潜力。