BriefGPT.xyz
Apr, 2024
LoRA专家混合模型
Mixture of LoRA Experts
HTML
PDF
Xun Wu, Shaohan Huang, Furu Wei
TL;DR
引入了Mixture of LoRA Experts (MoLE)方法,利用分层控制和无限制的分支选择,实现了对LoRA的优化融合性能和弹性组合能力的提升。通过在自然语言处理(NLP)和视觉与语言(V&L)领域进行广泛的实验评估,证实了MoLE的有效性。
Abstract
lora
has gained widespread acceptance in the
fine-tuning
of large pre-trained models to cater to a diverse array of
downstream tasks
, show
→