Minimum Bayesian Risk Decoding (MBR) emerges as a promising decoding algorithm in Neural Machine Translation. However, MBR performs poorly with label smoothing, which is surprising as label smoothing provides decent improvement with beam search and improves generality in various tasks. In this work, we show that the issue arises from the un-consistency of label smoothing on the token-level and sequence-level distributions. We demonstrate that even though label smoothing only causes a slight change in the token-level, the sequence-level distribution is highly skewed. We coin the issue \emph{distributional over-smoothness}. To address this issue, we propose a simple and effective method, Distributional Cooling MBR (DC-MBR), which manipulates the entropy of output distributions by tuning down the Softmax temperature. We theoretically prove the equivalence between pre-tuning label smoothing factor and distributional cooling. Experiments on NMT benchmarks validate that distributional cooling improves MBR's efficiency and effectiveness in various settings.

本文提出了一种名为 Distributional Cooling MBR 的简单有效的算法来解决最小贝叶斯风险解码（MBR）算法在标签平滑技术下表现不佳的问题，该问题是由标签平滑在标记级别和序列级别分布上不一致引起的，并且在 NMT 基准测试中证明了分布式冷却可以提高 MBR 在各种设置下的效率和有效性。

DC-MBR: 最小贝叶斯风险解码的分布式冷却