BriefGPT.xyz
Feb, 2024
贝叶斯参数高效微调克服灾难性遗忘
Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting
HTML
PDF
Haolin Chen, Philip N. Garner
TL;DR
通过使用贝叶斯学习技术和Laplace逼近,我们展示了在更通用的参数高效微调(PEFT)框架中如何解决灾难性遗忘问题,并比较了使用对角和克罗内克逼近的方法在预训练知识保留上的性能差异。
Abstract
Although motivated by the adaptation of text-to-speech synthesis models, we argue that more generic
parameter-efficient fine-tuning
(PEFT) is an appropriate framework to do such adaptation. However,
catastrophic forgett
→