Structured science summaries or research contributions using properties or
dimensions beyond traditional keywords enhances science findability. Current
methods, such as those used by the Open Research Knowledge Graph (ORKG),
involve manually curating properties to describe research papers' contributions
in a structured manner, but this is labor-intensive and inconsistent between
the domain expert human curators. We propose using Large Language Models (LLMs)
to automatically suggest these properties. However, it's essential to assess
the readiness of LLMs like GPT-3.5, Llama 2, and Mistral for this task before
application. Our study performs a comprehensive comparative analysis between
ORKG's manually curated properties and those generated by the aforementioned
state-of-the-art LLMs. We evaluate LLM performance through four unique
perspectives: semantic alignment and deviation with ORKG properties,
fine-grained properties mapping accuracy, SciNCL embeddings-based cosine
similarity, and expert surveys comparing manual annotations with LLM outputs.
These evaluations occur within a multidisciplinary science setting. Overall,
LLMs show potential as recommendation systems for structuring science, but
further finetuning is recommended to improve their alignment with scientific
tasks and mimicry of human expertise.

利用大型语言模型（LLMs）自动推荐属性，以增强科学论文的可发现性，并通过与 Open Research Knowledge Graph（ORKG）的比较分析评估 LLMs 的性能，发现 LLMs 在构建科学结构化推荐系统方面具有潜力，但需要进一步优化以提高与科学任务的一致性和模拟人类专业知识。