Recent work in cross-lingual contextual word embedding learning cannot handle
multi-sense words well. In this work, we explore the characteristics of
contextual word embeddings and show the link between contextual word embeddings
and word senses. We propose two improving solutions by considering contextual
multi-sense word embeddings as noise (removal) and by generating cluster level
average anchor embeddings for contextual multi-sense word embeddings
(replacement). Experiments show that our solutions can improve the supervised
contextual word embeddings alignment for multi-sense words in a microscopic
perspective without hurting the macroscopic performance on the bilingual
lexicon induction task. For unsupervised alignment, our methods significantly
improve the performance on the bilingual lexicon induction task for more than
10 points.

本文提出两种改进的解决方案，通过将上下文多义词嵌入视为噪声（去除）和通过生成聚类级别的平均锚嵌入以替换上下文多义词嵌入，提高了跨语言语境下上下文词嵌入对齐的微观性能，同时不会损害双语词汇表归纳任务的宏观性能；对于无监督对齐，我们的方法在双语词汇表归纳任务中显著提高了 10 多分。