This paper presents an exhaustive quantitative and qualitative evaluation of
Large Language Models (LLMs) for Knowledge Graph (KG) construction and
reasoning. We employ eight distinct datasets that encompass aspects including
entity, relation and event extraction, link prediction, and question answering.
Empirically, our findings suggest that GPT-4 outperforms ChatGPT in the
majority of tasks and even surpasses fine-tuned models in certain reasoning and
question-answering datasets. Moreover, our investigation extends to the
potential generalization ability of LLMs for information extraction, which
culminates in the presentation of the Virtual Knowledge Extraction task and the
development of the VINE dataset. Drawing on these empirical findings, we
further propose AutoKG, a multi-agent-based approach employing LLMs for KG
construction and reasoning, which aims to chart the future of this field and
offer exciting opportunities for advancement. We anticipate that our research
can provide invaluable insights for future undertakings of KG\footnote{Code and
datasets will be available in this https URL

本文对大型语言模型在知识图谱构建和推理方面进行了全面的定量和定性评估，结果表明，GPT-4 在大多数任务中表现优异，并在某些推理和问答数据集中甚至超过了微调模型。同时，我们还提出了基于多智能体的 AutoKG 方法，该方法可利用 LLMs 进行知识图谱构建和推理，为该领域的未来发展提供有趣的机遇。