Previous work on cross-lingual sequence labeling tasks either requires
parallel data or bridges the two languages through word-byword matching. Such
requirements and assumptions are infeasible for most languages, especially for
languages with large linguistic distances, e.g., English and Chinese. In this
work, we propose a Multilingual Language Model with deep semantic Alignment
(MLMA) to generate language-independent representations for cross-lingual
sequence labeling. Our methods require only monolingual corpora with no
bilingual resources at all and take advantage of deep contextualized
representations. Experimental results show that our approach achieves new
state-of-the-art NER and POS performance across European languages, and is also
effective on distant language pairs such as English and Chinese.

本文提出了多语言语言模型与深度语义对准（MLMA），用于产生跨语言标注的语言无关表示，方法不需要平行数据或一个词一个词匹配，只需要单语语料库，并利用深度上下文表示。实验结果表明，本方法在欧洲语言以及英语和汉语等远距离语言对上实现了新的最新 NER 和 POS 性能。