Biomedical literature is a rapidly expanding field of science and technology.
Classification of biomedical texts is an essential part of biomedicine
research, especially in the field of biology. This work proposes the fine-tuned
DistilBERT, a methodology-specific, pre-trained generative classification
language model for mining biomedicine texts. The model has proven its
effectiveness in linguistic understanding capabilities and has reduced the size
of BERT models by 40\% but by 60\% faster. The main objective of this project
is to improve the model and assess the performance of the model compared to the
non-fine-tuned model. We used DistilBert as a support model and pre-trained on
a corpus of 32,000 abstracts and complete text articles; our results were
impressive and surpassed those of traditional literature classification methods
by using RNN or LSTM. Our aim is to integrate this highly specialised and
specific model into different research industries.

生物医学文献是一个迅速扩展的科学技术领域，本研究提出了 Fine-tuned DistilBERT 方法，这是一种专门针对生物医学文本的预训练生成分类语言模型，通过 40% 减小 BERT 模型尺寸和 60% 的速度提升，证明其在语言理解能力上的有效性。本论文的主要目标是改进模型并评估其性能，以与非 Fine-tuned 模型进行对比。我们使用 DistilBERT 作为支持模型，在包括 32,000 篇摘要和完整文本文章的语料库上进行了预训练，其结果令人印象深刻，超过了使用 RNN 或 LSTM 的传统文献分类方法。我们的目标是将这个高度专业和特定的模型整合到不同的研究行业中。