Homographs, words with the same spelling but different meanings, remain
challenging in Neural Machine Translation (NMT). While recent works leverage
various word embedding approaches to differentiate word sense in NMT, they do
not focus on the pivotal components in resolving ambiguities of homographs in
NMT: the hidden states of an encoder. In this paper, we propose a novel
approach to tackle homographic issues of NMT in the latent space. We first
train an encoder (aka "HDR-encoder") to learn universal sentence
representations in a natural language inference (NLI) task. We further
fine-tune the encoder using homograph-based synset sentences from WordNet,
enabling it to learn word-level homographic disambiguation representations
(HDR). The pre-trained HDR-encoder is subsequently integrated with a
transformer-based NMT in various schemes to improve translation accuracy.
Experiments on four translation directions demonstrate the effectiveness of the
proposed method in enhancing the performance of NMT systems in the BLEU scores
(up to +2.3 compared to a solid baseline). The effects can be verified by other
metrics (F1, precision, and recall) of translation accuracy in an additional
disambiguation task. Visualization methods like heatmaps, T-SNE and translation
examples are also utilized to demonstrate the effects of the proposed method.

本文提出一种新的方法来解决神经机器翻译中同形异义词的问题，该方法利用编码器的隐藏状态以及受 WordNet 数据集启发的同义词句子进行 encoder 的微调，从而实现单词级别的同形异义词消歧表示（HDR），最终应用于 transformer-based NMT 以提高 BLEU 得分和其他翻译准确度指标。

学习同形异义词辨析表示用于神经机器翻译

Learning Homographic Disambiguation Representation for Neural Machine  Translation

The discrepancy between maximum likelihood estimation (MLE) and task measures
such as BLEU score has been studied before for autoregressive neural machine
translation (NMT) and resulted in alternative training algorithms (Ranzato et
al., 2016; Norouzi et al., 2016; Shen et al., 2016; Wu et al., 2018). However,
MLE training remains the de facto approach for autoregressive NMT because of
its computational efficiency and stability. Despite this mismatch between the
training objective and task measure, we notice that the samples drawn from an
MLE-based trained NMT support the desired distribution -- there are samples
with much higher BLEU score comparing to the beam decoding output. To benefit
from this observation, we train an energy-based model to mimic the behavior of
the task measure (i.e., the energy-based model assigns lower energy to samples
with higher BLEU score), which is resulted in a re-ranking algorithm based on
the samples drawn from NMT: energy-based re-ranking (EBR). We use both marginal
energy models (over target sentence) and joint energy models (over both source
and target sentences). Our EBR with the joint energy model consistently
improves the performance of the Transformer-based NMT: +4 BLEU points on
IWSLT'14 German-English, +3.0 BELU points on Sinhala-English, +1.2 BLEU on
WMT'16 English-German tasks.

通过基于能量的重新排序算法，利用基于 MLE 的 NMT 生成更高 BLEU 分数的样本，实现了比 MLE 更好的性能。

基于能量的再排序：利用能量模型来提高神经机器翻译的效果

Energy-Based Reranking: Improving Neural Machine Translation Using  Energy-Based Models

State-of-the-art Transformer-based neural machine translation (NMT) systems
still follow a standard encoder-decoder framework, in which source sentence
representation can be well done by an encoder with self-attention mechanism.
Though Transformer-based encoder may effectively capture general information in
its resulting source sentence representation, the backbone information, which
stands for the gist of a sentence, is not specifically focused on. In this
paper, we propose an explicit sentence compression method to enhance the source
sentence representation for NMT. In practice, an explicit sentence compression
goal used to learn the backbone information in a sentence. We propose three
ways, including backbone source-side fusion, target-side fusion, and both-side
fusion, to integrate the compressed sentence into NMT. Our empirical tests on
the WMT English-to-French and English-to-German translation tasks show that the
proposed sentence compression method significantly improves the translation
performances over strong baselines.

该研究提出了一种明确的句子压缩方法来增强源句子对于 NMT 的表征，包括背景信息，提出三种整合压缩句子到 NMT 的方法，并在 WMT 英法和英德翻译任务上进行实验测试，结果显示该压缩方法显著提高了翻译表现。