BriefGPT.xyz
Feb, 2025
低资源微调的联合定位与激活编辑
Joint Localization and Activation Editing for Low-Resource Fine-Tuning
HTML
PDF
Wen Lai, Alexander Fraser, Ivan Titov
TL;DR
本研究解决了在低资源场景中标准参数高效微调方法效果受限的问题。提出的联合定位与激活编辑(JoLA)方法,通过联合学习需要编辑的变换器头部、干预方式及其参数,实现了更有效的激活编辑。实验结果表明,JoLA在多个基准测试中表现优于现有方法,展现出其潜在影响力。
Abstract
Parameter-Efficient Fine-Tuning
(PEFT) methods, such as LoRA, are commonly used to adapt LLMs. However, the effectiveness of standard PEFT methods is limited in
Low-Resource Scenarios
with only a few hundred exam
→