Recent large language models (LLMs) can generate and revise text with
human-level performance, and have been widely commercialized in systems like
ChatGPT. These models come with clear limitations: they can produce inaccurate
information, reinforce existing biases, and be easily misused. Yet, many
scientists have been using them to assist their scholarly writing. How
wide-spread is LLM usage in the academic literature currently? To answer this
question, we use an unbiased, large-scale approach, free from any assumptions
on academic LLM usage. We study vocabulary changes in 14 million PubMed
abstracts from 2010-2024, and show how the appearance of LLMs led to an abrupt
increase in the frequency of certain style words. Our analysis based on excess
words usage suggests that at least 10% of 2024 abstracts were processed with
LLMs. This lower bound differed across disciplines, countries, and journals,
and was as high as 30% for some PubMed sub-corpora. We show that the appearance
of LLM-based writing assistants has had an unprecedented impact in the
scientific literature, surpassing the effect of major world events such as the
Covid pandemic.

LLM 的出现对科学文献产生了前所未有的影响，超过了像 Covid 大流行等重大世界事件的影响，至少有 10% 的 2024 篇摘要使用了 LLMs 处理。

通过过多词汇探索 ChatGPT 在学术写作中的应用

Delving into ChatGPT usage in academic writing through excess vocabulary

We consider the situation in which a user has collected a small set of
documents on a cohesive topic, and they want to retrieve additional documents
on this topic from a large collection. Information Retrieval (IR) solutions
treat the document set as a query, and look for similar documents in the
collection. We propose to extend the IR approach by treating the problem as an
instance of positive-unlabeled (PU) learning -- i.e., learning binary
classifiers from only positive and unlabeled data, where the positive data
corresponds to the query documents, and the unlabeled data is the results
returned by the IR engine. Utilizing PU learning for text with big neural
networks is a largely unexplored field. We discuss various challenges in
applying PU learning to the setting, including an unknown class prior,
extremely imbalanced data and large-scale accurate evaluation of models, and we
propose solutions and empirically validate them. We demonstrate the
effectiveness of the method using a series of experiments of retrieving PubMed
abstracts adhering to fine-grained topics. We demonstrate improvements over the
base IR solution and other baselines.

本研究提出了一种将信息检索 (IR) 扩展为正无标记 (positive-unlabeled, PU) 学习的方法，即从正样本数据 (即查询文档) 和非标记数据 (IR 引擎返回的结果) 中学习二元分类器，然后应用于文本分类问题中，该方法在 PubMed 文摘的细粒度主题提取中表现有效。

通过神经元正无标记学习实现文档集扩展的可扩展性评估和改进

Scalable Evaluation and Improvement of Document Set Expansion via Neural  Positive-Unlabeled Learning

Automatic identification and expansion of ambiguous abbreviations are
essential for biomedical natural language processing applications, such as
information retrieval and question answering systems. In this paper, we present
DEep Contextualized Biomedical. Abbreviation Expansion (DECBAE) model. DECBAE
automatically collects substantial and relatively clean annotated contexts for
950 ambiguous abbreviations from PubMed abstracts using a simple heuristic.
Then it utilizes BioELMo to extract the contextualized features of words, and
feed those features to abbreviation-specific bidirectional LSTMs, where the
hidden states of the ambiguous abbreviations are used to assign the exact
definitions. Our DECBAE model outperforms other baselines by large margins,
achieving average accuracy of 0.961 and macro-F1 of 0.917 on the dataset. It
also surpasses human performance for expanding a sample abbreviation, and
remains robust in imbalanced, low-resources and clinical settings.

本文介绍了一种名为 DEep Contextualized Biomedical Abbreviation Expansion（DECBAE）模型，该模型通过普通医学期刊摘要，利用简单启发式方法自动收集 950 个模棱两可的缩略语的上下文，并使用 BioELMo 从中提取单词的上下文化特征，将这些特征反馈给特定于缩略语的双向 LSTMs，以分配确切的定义。该模型在各项指标上均胜过其他基线模型和人类表现，并在不平衡、低资源和临床设置下保持稳定。