BriefGPT.xyz
Apr, 2020
通过不间断学习探索预训练跨语言模型的微调技巧
Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning
HTML
PDF
Zihan Liu, Genta Indra Winata, Andrea Madotto, Pascale Fung
TL;DR
针对fine-tuning预训练语言模型后其跨语言能力减弱的问题，该研究提出了一种利用continual learning来保持其原有跨语言能力的方法，并在句子检索、跨语言词性标注和命名实体识别等任务中达到更好的性能。
Abstract
Recently,
fine-tuning
pre-trained cross-lingual models (e.g., multilingual BERT) to downstream
cross-lingual tasks
has shown promising results. However, the
→