We show that language model finetuning can be improved, sometimes
dramatically, with a simple augmentation. NEFTune adds noise to the embedding
vectors during training. Standard finetuning of LLaMA-2-7B using Alpaca
achieves 29.79% on AlpacaEval, which rises to 64.69% using noisy embeddings.
NEFTune also improves over strong baselines on modern instruction datasets.
Models trained with Evol-Instruct see a 10% improvement, with ShareGPT an 8%
improvement, and with OpenPlatypus an 8% improvement. Even powerful models
further refined with RLHF such as LLaMA-2-Chat benefit from additional training
with NEFTune.

使用简单的增强方法 NEFTune，对语言模型进行细调可以显著提高性能，在 AlpacaEval 测试中，LLaMA-2-7B 的标准细调结果为 29.79%，而使用带有噪声的嵌入向量增强后提高至 64.69%。NEFTune 还在现代指令数据集上改进了强基准模型，在 Evol-Instruct 数据集上提高了 10%，在 ShareGPT 上提高了 8%，在 OpenPlatypus 上也提高了 8%。对于使用 RLHF 进一步优化的强大模型（如 LLaMA-2-Chat），通过 NEFTune 进行额外训练也带来了益处。