A backbone of knowledge graphs are their class membership relations, which
assign entities to a given class. As part of the knowledge engineering process,
we propose a new method for evaluating the quality of these relations by
processing descriptions of a given entity and class using a zero-shot
chain-of-thought classifier that uses a natural language intensional definition
of a class. We evaluate the method using two publicly available knowledge
graphs, Wikidata and CaLiGraph, and 7 large language models. Using the
gpt-4-0125-preview large language model, the method's classification
performance achieves a macro-averaged F1-score of 0.830 on data from Wikidata
and 0.893 on data from CaLiGraph. Moreover, a manual analysis of the
classification errors shows that 40.9% of errors were due to the knowledge
graphs, with 16.0% due to missing relations and 24.9% due to incorrectly
asserted relations. These results show how large language models can assist
knowledge engineers in the process of knowledge graph refinement. The code and
data are available on Github.

使用零 - shot 思维链分类器评估知识图谱的类成员关系质量，并使用两个公开知识图谱和七个大型语言模型进行评估，结果表明大型语言模型可以帮助知识工程师在知识图谱细化过程中进行知识图谱的改进。