Multilingual pretrained representations generally rely on subword
segmentation algorithms to create a shared multilingual vocabulary. However,
standard heuristic algorithms often lead to sub-optimal segmentation,
especially for languages with limited amounts of data. In this paper, we take
two major steps towards alleviating this problem. First, we demonstrate
empirically that applying existing subword regularization methods(Kudo, 2018;
Provilkov et al., 2020) during fine-tuning of pre-trained multilingual
representations improves the effectiveness of cross-lingual transfer. Second,
to take full advantage of different possible input segmentations, we propose
Multi-view Subword Regularization (MVR), a method that enforces the consistency
between predictions of using inputs tokenized by the standard and probabilistic
segmentations. Results on the XTREME multilingual benchmark(Hu et al., 2020)
show that MVR brings consistent improvements of up to 2.5 points over using
standard segmentation algorithms.

应用现有子词规范化方法 (Kudo,2018;Profilkov et al., 2020) 对预训练的多语言表示进行微调，通过多视角子词规范化（MVR）方法增强跨语言转移的有效性，同时改善标准分词算法的效果，提高多语种基准测试 (XTREME) 的一致性改进 2.5 个点。