Recent advancements in natural language processing, especially the emergence
of Large Language Models (LLMs), have opened exciting possibilities for
constructing computational simulations designed to replicate human behavior
accurately. However, LLMs are complex statistical learners without
straightforward deductive rules, making them prone to unexpected behaviors. In
this study, we highlight the limitations of LLMs in simulating human
interactions, particularly focusing on LLMs' ability to simulate political
debates. Our findings indicate a tendency for LLM agents to conform to the
model's inherent social biases despite being directed to debate from certain
political perspectives. This tendency results in behavioral patterns that seem
to deviate from well-established social dynamics among humans. We reinforce
these observations using an automatic self-fine-tuning method, which enables us
to manipulate the biases within the LLM and demonstrate that agents
subsequently align with the altered biases. These results underscore the need
for further research to develop methods that help agents overcome these biases,
a critical step toward creating more realistic simulations.

近期自然语言处理的进展，特别是大型语言模型（LLMs）的出现，为构建精确复制人类行为的计算模拟提供了令人兴奋的可能性。然而，LLMs 是复杂的统计学习器，缺乏直接的演绎规则，因此容易产生意想不到的行为。本研究突出了 LLMs 在模拟人类互动方面的局限性，特别关注 LLMs 在模拟政治辩论方面的能力。我们的发现表明，尽管被指示从特定的政治角度进行辩论，LLM 代理倾向于符合模型固有的社会偏见。这种倾向导致了行为模式的偏离，似乎偏离了人类之间已经确立的社会动力学规律。我们使用了一种自动自我微调方法来强化这些观察结果，该方法使我们能够操纵 LLM 内的偏见，并展示代理随后与改变后的偏见保持一致。这些结果强调了进一步研究的必要性，以开发帮助代理克服这些偏见的方法，是创造更现实模拟的关键一步。

辩论中 LLM 仿真的系统偏差

Systematic Biases in LLM Simulations of Debates

People constantly use language to learn about the world. Computational
linguists have capitalized on this fact to build large language models (LLMs)
that acquire co-occurrence-based knowledge from language corpora. LLMs achieve
impressive performance on many tasks, but the robustness of their world
knowledge has been questioned. Here, we ask: do LLMs acquire generalized
knowledge about real-world events? Using curated sets of minimal sentence pairs
(n=1215), we tested whether LLMs are more likely to generate plausible event
descriptions compared to their implausible counterparts. We found that LLMs
systematically distinguish possible and impossible events (The teacher bought
the laptop vs. The laptop bought the teacher) but fall short of human
performance when distinguishing likely and unlikely events (The nanny tutored
the boy vs. The boy tutored the nanny). In follow-up analyses, we show that (i)
LLM scores are driven by both plausibility and surface-level sentence features,
(ii) LLMs generalize well across syntactic sentence variants (active vs
passive) but less well across semantic sentence variants (synonymous
sentences), (iii) some, but not all LLM deviations from ground-truth labels
align with crowdsourced human judgments, and (iv) explicit event plausibility
information emerges in middle LLM layers and remains high thereafter. Overall,
our analyses reveal a gap in LLMs' event knowledge, highlighting their
limitations as generalized knowledge bases. We conclude by speculating that the
differential performance on impossible vs. unlikely events is not a temporary
setback but an inherent property of LLMs, reflecting a fundamental difference
between linguistic knowledge and world knowledge in intelligent systems.

研究人员测试了大型语言模型是否能够生成可信的真实世界事件描述，并发现这些模型虽然能够区分可能和不可能的事件，但在区分可能性较小的事件时仍然比人类表现差，这突显了 LLM 在事件知识方面存在的局限性。