Researchers must stay current in their fields by regularly reviewing academic
literature, a task complicated by the daily publication of thousands of papers.
Traditional multi-label text classification methods often ignore semantic
relationships and fail to address the inherent class imbalances. This paper
introduces a novel approach using the SciBERT model and CNNs to systematically
categorize academic abstracts from the Elsevier OA CC-BY corpus. We use a
multi-segment input strategy that processes abstracts, body text, titles, and
keywords obtained via BERT topic modeling through SciBERT. Here, the [CLS]
token embeddings capture the contextual representation of each segment,
concatenated and processed through a CNN. The CNN uses convolution and pooling
to enhance feature extraction and reduce dimensionality, optimizing the data
for classification. Additionally, we incorporate class weights based on label
frequency to address the class imbalance, significantly improving the
classification F1 score and enhancing text classification systems and
literature review efficiency.

该论文介绍了一种使用 SciBERT 模型和 CNN 的新方法，通过处理摘要、正文、标题和关键词等多个部分，以解决多标签文本分类中的语义关系和类别不平衡问题，并在文献综述效率方面取得显著改进。

利用基于 BERT 的模型推动跨学科研究：通过 SciBERT-CNN 与主题建模的方法

Empowering Interdisciplinary Research with BERT-Based Models: An  Approach Through SciBERT-CNN with Topic Modeling

Clinical notes containing valuable patient information are written by
different health care providers with various scientific levels and writing
styles. It might be helpful for clinicians and researchers to understand what
information is essential when dealing with extensive electronic medical
records. Entities recognizing and mapping them to standard terminologies is
crucial in reducing ambiguity in processing clinical notes. Although named
entity recognition and entity linking are critical steps in clinical natural
language processing, they can also result in the production of repetitive and
low-value concepts. In other hand, all parts of a clinical text do not share
the same importance or content in predicting the patient's condition. As a
result, it is necessary to identify the section in which each content is
recorded and also to identify key concepts to extract meaning from clinical
texts. In this study, these challenges have been addressed by using clinical
natural language processing techniques. In addition, in order to identify key
concepts, a set of popular unsupervised key phrase extraction methods has been
verified and evaluated. Considering that most of the clinical concepts are in
the form of multi-word expressions and their accurate identification requires
the user to specify n-gram range, we have proposed a shortcut method to
preserve the structure of the expression based on TF-IDF. In order to evaluate
the pre-processing method and select the concepts, we have designed two types
of downstream tasks (multiple and binary classification) using the capabilities
of transformer-based models. The obtained results show the superiority of
proposed method in combination with SciBERT model, also offer an insight into
the efficacy of general extracting essential phrase methods for clinical notes.

在这项研究中，使用临床自然语言处理技术解决了识别和映射标准术语的问题。提出了一种用于提取临床文本中关键概念的 TF-IDF 快捷方法，并使用基于 transformer 的模型设计了两种下游任务，结果表明 SciBERT 模型在结合提出的方法时具有优越性，并为临床笔记的关键短语提取提供了洞察。