Modern NLP applications have enjoyed a great boost utilizing neural networks
models. Such deep neural models, however, are not applicable to most human
languages due to the lack of annotated training data for various NLP tasks.
Cross-lingual transfer learning (CLTL) is a viable method for building NLP
models for a low-resource target language by leveraging labeled data from other
(source) languages. In this work, we focus on the multilingual transfer setting
where training data in multiple source languages is leveraged to further boost
target language performance.
Unlike most existing methods that rely only on language-invariant features
for CLTL, our approach coherently utilizes both language-invariant and
language-specific features at instance level. Our model leverages adversarial
networks to learn language-invariant features, and mixture-of-experts models to
dynamically exploit the similarity between the target language and each
individual source language. This enables our model to learn effectively what to
share between various languages in the multilingual setup. Moreover, when
coupled with unsupervised multilingual embeddings, our model can operate in a
zero-resource setting where neither target language training data nor
cross-lingual resources are available. Our model achieves significant
performance gains over prior art, as shown in an extensive set of experiments
over multiple text classification and sequence tagging tasks including a
large-scale industry dataset.

利用生成对抗网络和无监督多语言嵌入，结合多源语言训练数据的多语言转移学习方法，在多语言文本分类和序列标记任务中实现 target 语言的显著性能提升。