This work aims toward an application of natural language processing (NLP)
technology for automatic verification of causal graphs using text sources. A
causal graph is often derived from unsupervised causal discovery methods and
requires manual evaluation from human experts. NLP technologies, i.e., Large
Language Models (LLMs) such as BERT and ChatGPT, can potentially be used to
verify the resulted causal graph by predicting if causal relation can be
observed between node pairs based on the textual context. In this work, we
compare the performance of two types of NLP models: (1) Pre-trained language
models fine-tuned for causal relation classification task and, (2) prompt-based
LLMs. Contrasted to previous studies where prompt-based LLMs work relatively
well over a set of diverse tasks, preliminary experiments on biomedical and
open-domain datasets suggest that the fine-tuned models far outperform the
prompt-based LLMs, up to 20.5 points improvement of F1 score. We shared the
code and the pre-processed datasets in our repository.

该研究旨在利用自然语言处理技术通过文本来源对因果图进行自动验证。与之前的研究成果相比，该研究发现经过精调的预训练语言模型在因果关系分类任务上表现远超基于提示的语言模型，F1 得分最高提高了 20.5 个百分点。