Parameter-efficient fine-tuning (PEFT) methods seek to adapt large models via
updates to a small number of weights. However, much prior interpretability work
has shown that representations encode rich semantic information, suggesting
that editing representations might be a more powerful alternative. Here, we
pursue this hypothesis by developing a family of $\textbf{Representation
Finetuning (ReFT)}$ methods. ReFT methods operate on a frozen base model and
learn task-specific interventions on hidden representations. We define a strong
instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT). LoReFT is
a drop-in replacement for existing PEFTs and learns interventions that are
10x-50x more parameter-efficient than prior state-of-the-art PEFTs. We showcase
LoReFT on eight commonsense reasoning tasks, four arithmetic reasoning tasks,
Alpaca-Eval v1.0, and GLUE. In all these evaluations, LoReFT delivers the best
balance of efficiency and performance, and almost always outperforms
state-of-the-art PEFTs. We release a generic ReFT training library publicly at
this https URL

参数高效微调方法通过少量权重的调整来适应大模型。本研究通过发展一系列表示微调方法来探索该假设，利用隐藏表示上的任务特定干预进行表示编辑，展示在多个评估中，低秩线性子空间表示微调方法可以实现比先前最先进参数高效微调方法更好的效率和性能平衡。