Traditional dataset retrieval systems index on metadata information rather
than on the data values. Thus relying primarily on manual annotations and
high-quality metadata, processes known to be labour-intensive and challenging
to automate. We propose a method to support metadata enrichment with topic
annotations of column headers using three Large Language Models (LLMs):
ChatGPT-3.5, GoogleBard and GoogleGemini. We investigate the LLMs ability to
classify column headers based on domain-specific topics from a controlled
vocabulary. We evaluate our approach by assessing the internal consistency of
the LLMs, the inter-machine alignment, and the human-machine agreement for the
topic classification task. Additionally, we investigate the impact of
contextual information (i.e. dataset description) on the classification
outcomes. Our results suggest that ChatGPT and GoogleGemini outperform
GoogleBard for internal consistency as well as LLM-human-alignment.
Interestingly, we found that context had no impact on the LLMs performances.
This work proposes a novel approach that leverages LLMs for text classification
using a controlled topic vocabulary, which has the potential to facilitate
automated metadata enrichment, thereby enhancing dataset retrieval and the
Findability, Accessibility, Interoperability and Reusability (FAIR) of research
data on the Web.

传统数据集检索系统主要依赖元数据信息进行索引，而不是数据值。本文提出了一种使用三个大型语言模型 (ChatGPT-3.5、GoogleBard 和 GoogleGemini) 对列标题进行主题注释的元数据增强方法。通过评估模型的内部一致性、机器间对齐性和人机一致性来验证模型的能力，并研究上下文信息对分类结果的影响。结果表明，ChatGPT 和 GoogleGemini 在内部一致性和模型 - 人的一致性方面优于 GoogleBard。有趣的是，我们发现上下文对于语言模型的表现没有影响。本文提出了一种使用控制主题词汇来利用语言模型进行文本分类的新方法，有潜力促进自动化的元数据增强，从而提升在 Web 上的数据集检索、查找性、可访问性、互操作性和可重用性 (FAIR)。