Recent advances in artificial intelligence have seen Large Language Models (LLMs) demonstrate notable proficiency in causal discovery tasks. This study explores the factors influencing the performance of LLMs in causal discovery tasks. Utilizing open-source LLMs, we examine how the frequency of causal relations within their pre-training corpora affects their ability to accurately respond to causal discovery queries. Our findings reveal that a higher frequency of causal mentions correlates with better model performance, suggesting that extensive exposure to causal information during training enhances the models' causal discovery capabilities. Additionally, we investigate the impact of context on the validity of causal relations. Our results indicate that LLMs might exhibit divergent predictions for identical causal relations when presented in different contexts. This paper provides the first comprehensive analysis of how different factors contribute to LLM performance in causal discovery tasks.

本文研究了影响大型语言模型（LLM）在因果发现任务中表现的因素，特别是预训练语料库中因果关系的频率如何影响模型的准确性。研究发现，因果提及频率越高，模型性能越好，表明训练过程中对因果信息的广泛接触提升了模型的因果发现能力。此外，文章还探讨了上下文对因果关系有效性的影响，揭示在不同上下文中相同因果关系可能导致模型的预测出现差异。

从预训练语料库到大型语言模型：哪些因素影响大型语言模型在因果发现任务中的表现？