Neural machine translation (NMT) has achieved new state-of-the-art
performance in translating ambiguous words. However, it is still unclear which
component dominates the process of disambiguation. In this paper, we explore
the ability of NMT encoders and decoders to disambiguate word senses by
evaluating hidden states and investigating the distributions of self-attention.
We train a classifier to predict whether a translation is correct given the
representation of an ambiguous noun. We find that encoder hidden states
outperform word embeddings significantly which indicates that encoders
adequately encode relevant information for disambiguation into hidden states.
Decoders could provide further relevant information for disambiguation.
Moreover, the attention weights and attention entropy show that self-attention
can detect ambiguous nouns and distribute more attention to the context. Note
that this is a revised version. The content related to decoder hidden states
has been updated.

本文探讨了 NMT 编码器和解码器通过评估隐藏状态和研究自我关注的分布来消歧义词义的能力。通过训练分类器并研究注意权重和注重熵，发现编码器的隐藏状态显著优于单词嵌入，并且自我关注能够检测到模糊不清的名词并将更多注意力分配给上下文。