Existing open-source helpfulness preference datasets do not specify what
makes some responses more helpful and others less so. Models trained on these
datasets can incidentally learn to model dataset artifacts (e.g. preferring
longer but unhelpful responses only due to their length). To alleviate this
problem, we collect HelpSteer, a multi-attribute helpfulness dataset annotated
for the various aspects that make responses helpful. Specifically, our
37k-sample dataset has annotations for correctness, coherence, complexity, and
verbosity in addition to overall helpfulness of responses. Training Llama 2 70B
using the HelpSteer dataset with SteerLM technique produces a model that scores
7.54 on MT Bench, which is currently the highest score for open models that do
not require training data from more powerful models (e.g. GPT4). We release
this dataset with CC-BY-4.0 license at
this https URL

通过使用 HelpSteer 数据集和 SteerLM 技术训练 Llama 2 70B 模型，我们在 MT Bench 上获得了 7.54 的分数，这是目前不需要更强大模型（如 GPT4）的训练数据的开放模型得分最高的。