Recently, the NLP community has witnessed a rapid advancement in multilingual
and cross-lingual transfer research where the supervision is transferred from
high-resource languages (HRLs) to low-resource languages (LRLs). However, the
cross-lingual transfer is not uniform across languages, particularly in the
zero-shot setting. Towards this goal, one promising research direction is to
learn shareable structures across multiple tasks with limited annotated data.
The downstream multilingual applications may benefit from such a learning setup
as most of the languages across the globe are low-resource and share some
structures with other languages. In this paper, we propose a novel
meta-learning framework (called Meta-X$_{NLG}$) to learn shareable structures
from typologically diverse languages based on meta-learning and language
clustering. This is a step towards uniform cross-lingual transfer for unseen
languages. We first cluster the languages based on language representations and
identify the centroid language of each cluster. Then, a meta-learning algorithm
is trained with all centroid languages and evaluated on the other languages in
the zero-shot setting. We demonstrate the effectiveness of this modeling on two
NLG tasks (Abstractive Text Summarization and Question Generation), 5 popular
datasets and 30 typologically diverse languages. Consistent improvements over
strong baselines demonstrate the efficacy of the proposed framework. The
careful design of the model makes this end-to-end NLG setup less vulnerable to
the accidental translation problem, which is a prominent concern in zero-shot
cross-lingual NLG tasks.

本文提出了一种基于元学习和语言聚类的元学习框架（Meta-X$_{NLG}$），从不同的语言中学习可共享的结构，以实现对未知语言的统一跨语种转移。在两个自动生成任务（提取型文本摘要和问题生成）以及 30 种类型各异的语言和 5 个广泛使用的数据集上，该框架展示了相对于强基线的一致的改进效果，并且该模型的细致设计使得该端到端自动化生成设置不太容易受到意外翻译问题的影响，这在零 - shot 跨语种自然语言生成任务中是一个突出的问题。

Meta-X_{NLG}: 一种基于语言聚类的元学习方法，用于零样本跨语言传输和生成

Meta-X$_{NLG}$: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation

While natural language processing systems often focus on a single language,
multilingual transfer learning has the potential to improve performance,
especially for low-resource languages. We introduce XLDA, cross-lingual data
augmentation, a method that replaces a segment of the input text with its
translation in another language. XLDA enhances performance of all 14 tested
languages of the cross-lingual natural language inference (XNLI) benchmark.
With improvements of up to $4.8\%$, training with XLDA achieves
state-of-the-art performance for Greek, Turkish, and Urdu. XLDA is in contrast
to, and performs markedly better than, a more naive approach that aggregates
examples in various languages in a way that each example is solely in one
language. On the SQuAD question answering task, we see that XLDA provides a
$1.0\%$ performance increase on the English evaluation set. Comprehensive
experiments suggest that most languages are effective as cross-lingual
augmentors, that XLDA is robust to a wide range of translation quality, and
that XLDA is even more effective for randomly initialized models than for
pretrained models.

介绍了一种名为 XLDA 的跨语言数据增强方法，该方法通过替换输入文本中的部分内容为另一种语言的翻译来改善跨语言自然语言推理任务的性能，在 14 种语言的测试中均获得了提升，并在希腊语、土耳其语和乌尔都语方面取得了最好的性能，且远优于其他方法。

XLDA：基于跨语言数据增强的自然语言推断与问答

XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and  Question Answering

Universal schema builds a knowledge base (KB) of entities and relations by
jointly embedding all relation types from input KBs as well as textual patterns
expressing relations from raw text. In most previous applications of universal
schema, each textual pattern is represented as a single embedding, preventing
generalization to unseen patterns. Recent work employs a neural network to
capture patterns' compositional semantics, providing generalization to all
possible input text. In response, this paper introduces significant further
improvements to the coverage and flexibility of universal schema relation
extraction: predictions for entities unseen in training and multilingual
transfer learning to domains with no annotation. We evaluate our model through
extensive experiments on the English and Spanish TAC KBP benchmark,
outperforming the top system from TAC 2013 slot-filling using no handwritten
patterns or additional annotation. We also consider a multilingual setting in
which English training data entities overlap with the seed KB, but Spanish text
does not. Despite having no annotation for Spanish data, we train an accurate
predictor, with additional improvements obtained by tying word embeddings
across languages. Furthermore, we find that multilingual training improves
English relation extraction accuracy. Our approach is thus suited to
broad-coverage automated knowledge base construction in a variety of languages
and domains.

本文介绍了通用模式中关系提取的重大进展，包含对实体预测，跨语言迁移学习以及在没有标注数据的情况下进行准确预测等方面的改进，展示出其在多语言和领域中的适用性，以及在英文数据集中关系提取准确性的进一步提升。