LLM-based auto-annotators have become a key component of the LLM development
process due to their cost-effectiveness and scalability compared to human-based
evaluation. However, these auto-annotators can introduce complex biases that
are hard to remove. Even simple, known confounders such as preference for
longer outputs remain in existing automated evaluation metrics. We propose a
simple regression analysis approach for controlling biases in auto-evaluations.
As a real case study, we focus on reducing the length bias of AlpacaEval, a
fast and affordable benchmark for chat LLMs that uses LLMs to estimate response
quality. Despite being highly correlated with human preferences, AlpacaEval is
known to favor models that generate longer outputs. We introduce a
length-controlled AlpacaEval that aims to answer the counterfactual question:
"What would the preference be if the model's and baseline's output had the same
length?". To achieve this, we first fit a generalized linear model to predict
the biased output of interest (auto-annotator preferences) based on the
mediators we want to control for (length difference) and other relevant
features. We then obtain length-controlled preferences by predicting
preferences while conditioning the GLM with a zero difference in lengths.
Length-controlling not only improves the robustness of the metric to
manipulations in model verbosity, we also find that it increases the Spearman
correlation with LMSYS' Chatbot Arena from 0.94 to 0.98. We release the code
and leaderboard at this https URL .

通过回归分析控制偏见，减少 AlpacaEval 的长度偏差，提高与 Chatbot Arena 的相关性。

长度可控的 AlpacaEval：消除自动评估偏见的简单方法

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic  Evaluators

We show that language model finetuning can be improved, sometimes
dramatically, with a simple augmentation. NEFTune adds noise to the embedding
vectors during training. Standard finetuning of LLaMA-2-7B using Alpaca
achieves 29.79% on AlpacaEval, which rises to 64.69% using noisy embeddings.
NEFTune also improves over strong baselines on modern instruction datasets.
Models trained with Evol-Instruct see a 10% improvement, with ShareGPT an 8%
improvement, and with OpenPlatypus an 8% improvement. Even powerful models
further refined with RLHF such as LLaMA-2-Chat benefit from additional training
with NEFTune.

使用简单的增强方法 NEFTune，对语言模型进行细调可以显著提高性能，在 AlpacaEval 测试中，LLaMA-2-7B 的标准细调结果为 29.79%，而使用带有噪声的嵌入向量增强后提高至 64.69%。NEFTune 还在现代指令数据集上改进了强基准模型，在 Evol-Instruct 数据集上提高了 10%，在 ShareGPT 上提高了 8%，在 OpenPlatypus 上也提高了 8%。对于使用 RLHF 进一步优化的强大模型（如 LLaMA-2-Chat），通过 NEFTune 进行额外训练也带来了益处。