Jul, 2024
通过慢速级联学习实现富表达和可推广的大规模模型的低秩适应
Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning
Siwei Li, Yifan Yang, Yifei Shen, Fangyun Wei, Zongqing Lu...
TL;DRLoRA Slow Cascade Learning (LoRASC) is an innovative technique that enhances the expressiveness and generalization capabilities of efficient fine-tuning through a cascaded learning strategy and a slow-fast update mechanism, improving model stability, mitigating overfitting, and enhancing OOD robustness.