Despite being trained specifically to follow user instructions, today's
language models perform poorly when instructed to produce random outputs. For
example, when prompted to pick a number uniformly between one and ten
Llama-2-13B-chat disproportionately favors the number five, and when tasked
with picking a first name at random, Mistral-7B-Instruct chooses Avery 40 times
more often than we would expect based on the U.S. population. When these
language models are used for real-world tasks where diversity of outputs is
crucial, such as language model assisted dataset construction, their inability
to produce diffuse distributions over valid choices is a major hurdle. In this
work, we propose a fine-tuning method that encourages language models to output
distributions that are diffuse over valid outcomes. The methods we introduce
generalize across a variety of tasks and distributions and make large language
models practical for synthetic dataset generation with little human
intervention.

今天的语言模型在随机输出方面表现不佳，本文提出了一种微调方法，鼓励语言模型产生分散的输出分布，使大型语言模型在合成数据集生成方面变得更实用。

强制语言模型排除扩散分布

Forcing Diffuse Distributions out of Language Models

We analyze egocentric views of attended objects from infants. This paper
shows 1) empirical evidence that children's egocentric views have more diverse
distributions compared to adults' views, 2) we can computationally simulate the
infants' distribution, and 3) the distribution is beneficial for training more
generalized image classifiers not only for infant egocentric vision but for
third-person computer vision.

分析婴儿视角的自我中心视图，研究发现儿童自我中心视图分布比成年人更为多样化，通过计算模拟该分布并证明其有益于培养更广泛的图像分类器。