The exponential growth of online textual content across diverse domains has necessitated advanced methods for automated text classification. Large Language Models (LLMs) based on transformer architectures have shown significant success in this area, particularly in natural language processing (NLP) tasks. However, general-purpose LLMs often struggle with domain-specific content, such as scientific texts, due to unique challenges like specialized vocabulary and imbalanced data. In this study, we fine-tune four state-of-the-art LLMs BERT, SciBERT, BioBERT, and BlueBERT on three datasets derived from the WoS-46985 dataset to evaluate their performance in scientific text classification. Our experiments reveal that domain-specific models, particularly SciBERT, consistently outperform general-purpose models in both abstract-based and keyword-based classification tasks. Additionally, we compare our achieved results with those reported in the literature for deep learning models, further highlighting the advantages of LLMs, especially when utilized in specific domains. The findings emphasize the importance of domain-specific adaptations for LLMs to enhance their effectiveness in specialized text classification tasks.

本研究针对大型语言模型（LLMs）在科学文本分类中面临的挑战，特别是领域特定内容的处理困难，通过微调BERT、SciBERT、BioBERT和BlueBERT这四个模型，探讨其在科学文本分类中的表现。研究发现，领域特定模型SciBERT在分类任务中始终表现优于通用模型，强调了领域适配对提升LLMs在专业文本分类任务中的有效性的关键作用。

针对科学文本分类的大型语言模型微调：一项比较研究