Despite being trained specifically to follow user instructions, today's
language models perform poorly when instructed to produce random outputs. For
example, when prompted to pick a number uniformly between one and ten
Llama-2-13B-chat disproportionately favors the number five, and when tasked
with picking a first name at random, Mistral-7B-Instruct chooses Avery 40 times
more often than we would expect based on the U.S. population. When these
language models are used for real-world tasks where diversity of outputs is
crucial, such as language model assisted dataset construction, their inability
to produce diffuse distributions over valid choices is a major hurdle. In this
work, we propose a fine-tuning method that encourages language models to output
distributions that are diffuse over valid outcomes. The methods we introduce
generalize across a variety of tasks and distributions and make large language
models practical for synthetic dataset generation with little human
intervention.

今天的语言模型在随机输出方面表现不佳，本文提出了一种微调方法，鼓励语言模型产生分散的输出分布，使大型语言模型在合成数据集生成方面变得更实用。