BriefGPT.xyz
May, 2024
VB-LoRA:基于向量库的极度参数高效的微调
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
HTML
PDF
Yang Li, Shaobo Han, Shihao Ji
TL;DR
通过使用“divide-and-share”范式并引入向量库,VB-LoRA实现了极高的参数效率同时保持与当前最先进PEFT方法相当甚至更好的性能。
Abstract
As the adoption of large language models increases and the need for per-user or per-task model customization grows, the
parameter-efficient fine-tuning
(PEFT) methods, such as
low-rank adaptation
(LoRA) and its v
→