BriefGPT.xyz
Apr, 2024
HFT: 大型语言模型的半微调
HFT: Half Fine-Tuning for Large Language Models
HTML
PDF
Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Weiran Xu, Yu Sun...
TL;DR
通过定期重置部分参数,半精调可以恢复一些原始知识,并且在大规模语言模型中减轻了遗忘问题,同时在一系列下游基准测试中取得了最佳性能。
Abstract
large language models
(LLMs) with one or more
fine-tuning
phases have become a necessary step to unlock various capabilities, enabling LLMs to follow natural language instructions or align with human preferences.
→