BriefGPT.xyz
Oct, 2022
具有语言特定子网络的数据有效跨语言转移
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks
HTML
PDF
Rochelle Choenni, Dan Garrette, Ekaterina Shutova
TL;DR
本文提出了一种在多语言模型中使用语言特定的子网络的新方法，以控制跨语言参数共享，减少冲突，并在微调过程中增加正向迁移能力，结合元学习技术进行优化，通过广泛的分析验证了方法对模型的影响。
Abstract
Large
multilingual language models
typically share their parameters across all languages, which enables
cross-lingual task transfer
, but learning can also be hindered when training updates from different language
→