Cross-lingual Machine Reading Comprehension (CLMRC) remains a challenging
problem due to the lack of large-scale annotated datasets in low-source
languages, such as Arabic, Hindi, and Vietnamese. Many previous approaches use
translation data by translating from a rich-source language, such as English,
to low-source languages as auxiliary supervision. However, how to effectively
leverage translation data and reduce the impact of noise introduced by
translation remains onerous. In this paper, we tackle this challenge and
enhance the cross-lingual transferring performance by a novel augmentation
approach named Language Branch Machine Reading Comprehension (LBMRC). A
language branch is a group of passages in one single language paired with
questions in all target languages. We train multiple machine reading
comprehension (MRC) models proficient in individual language based on LBMRC.
Then, we devise a multilingual distillation approach to amalgamate knowledge
from multiple language branch models to a single model for all target
languages. Combining the LBMRC and multilingual distillation can be more robust
to the data noises, therefore, improving the model's cross-lingual ability.
Meanwhile, the produced single multilingual model is applicable to all target
languages, which saves the cost of training, inference, and maintenance for
multiple models. Extensive experiments on two CLMRC benchmarks clearly show the
effectiveness of our proposed method.

本文提出了一种名为 LBMRC 的创新增强方法和一种多语言蒸馏方法，共同提高跨语言转移性能，并且可以更加鲁棒地处理数据噪声，同时我们在两个 CLMRC 基准测试上进行了广泛的实验并验证了我们的方法的有效性。