In this research study, we empirically investigate the effect of sampling
temperature on the performance of Large Language Models (LLMs) on various
problem-solving tasks. We created a multiple-choice question-and-answer (MCQA)
exam by randomly sampling problems from standard LLM benchmarks. Then, we used
four popular LLMs with five prompt-engineering techniques to solve the MCQA
problems while increasing the sampling temperature from 0.0 to 1.0. Despite
anecdotal reports to the contrary, our empirical results indicate that changes
in temperature in the range 0.0 to 1.0 do not have a statistically significant
impact on LLM performance for problem-solving tasks. In addition, these results
appear to hold regardless of the LLM, the prompt-engineering technique, or the
problem domain. All code, data, and supplemental materials are available on
GitHub at: this https URL

通过对大型语言模型的不同采样温度进行实证研究，发现采样温度在 0.0 至 1.0 范围内对于解决问题的性能没有统计学上显著的影响。

采样温度对大型语言模型的问题解决效果的影响

The Effect of Sampling Temperature on Problem Solving in Large Language  Models

There have been wide spread claims in the literature about the emergent
reasoning capabilities of Pretrained Large Language Models. However, recent
studies, have found that their ability to plan remains questionable. Through
our experiments using GPT-2, we empirically demonstrate that the performance of
a finetuned baseline remains poor because it violates pre-conditions of actions
in the plans that it generates. To improve the planning capabilities of a
finetuned LLM, we train a verifier, which can classify actions as being valid
or invalid in a particular state. By randomly sampling actions from the same
dataset, we generate examples of invalid actions which are then used to train a
verifier which can check for action applicability. In the presence of diverse
sampling from a generator and a verifier which can prune invalid trajectories,
we show significant gains in the success rate on the Blocksworld domain.
Additionally, we show that finetuning the GPT-2 generator itself to create the
verifier generalizes better than finetuning the base GPT-2. Lastly, we
investigate the role of the sampling temperature which can be used to control
the exploration-exploitation tradeoff.

本论文通过使用 GPT-2 实验来证明，预训练的大型语言模型在计划方面的表现较差，研究人员首先建立了一个验证器在特定状态下对行动的适用性进行分类，然后在生成器中随机抽样无效动作来训练验证器，在生成器和验证器的共同作用下，取得了不错的成果。

利用验证器提高预训练语言模型的规划能力

Learning and Leveraging Verifiers to Improve Planning Capabilities of  Pre-trained Language Models

Multilingual machine translation models can benefit from synergy between
different language pairs, but also suffer from interference. While there is a
growing number of sophisticated methods that aim to eliminate interference, our
understanding of interference as a phenomenon is still limited. This work
identifies the main factors that contribute to interference in multilingual
machine translation. Through systematic experimentation, we find that
interference (or synergy) are primarily determined by model size, data size,
and the proportion of each language pair within the total dataset. We observe
that substantial interference occurs mainly when the model is very small with
respect to the available training data, and that using standard transformer
configurations with less than one billion parameters largely alleviates
interference and promotes synergy. Moreover, we show that tuning the sampling
temperature to control the proportion of each language pair in the data is key
to balancing the amount of interference between low and high resource language
pairs effectively, and can lead to superior performance overall.

本文介绍了多语言机器翻译中干扰现象的主要因素，以及如何通过控制模型大小、数据大小和采样温度来减轻干扰并提升性能。