Scientific publishing lays the foundation of science by disseminating
research findings, fostering collaboration, encouraging reproducibility, and
ensuring that scientific knowledge is accessible, verifiable, and built upon
over time. Recently, there has been immense speculation about how many people
are using large language models (LLMs) like ChatGPT in their academic writing,
and to what extent this tool might have an effect on global scientific
practices. However, we lack a precise measure of the proportion of academic
writing substantially modified or produced by LLMs. To address this gap, we
conduct the first systematic, large-scale analysis across 950,965 papers
published between January 2020 and February 2024 on the arXiv, bioRxiv, and
Nature portfolio journals, using a population-level statistical framework to
measure the prevalence of LLM-modified content over time. Our statistical
estimation operates on the corpus level and is more robust than inference on
individual instances. Our findings reveal a steady increase in LLM usage, with
the largest and fastest growth observed in Computer Science papers (up to
17.5%). In comparison, Mathematics papers and the Nature portfolio showed the
least LLM modification (up to 6.3%). Moreover, at an aggregate level, our
analysis reveals that higher levels of LLM-modification are associated with
papers whose first authors post preprints more frequently, papers in more
crowded research areas, and papers of shorter lengths. Our findings suggests
that LLMs are being broadly used in scientific writings.

通过对 950,965 篇论文进行大规模分析，我们发现大型语言模型在学术写作中的使用不断增加，尤其在计算机科学论文中增长最快（最高达 17.5%），而数学论文和 Nature 期刊的修改程度相对较低（最高达 6.3%）。此外，我们的研究发现，更多使用大型语言模型的论文往往由经常发布预印本的第一作者、研究领域较为拥挤的论文以及长度较短的论文撰写而成。这些结果表明大型语言模型在科学写作中被广泛应用。

科学论文中 LLM 的使用增加趋势的映射

Mapping the Increasing Use of LLMs in Scientific Papers

To this date, the efficacy of the scientific publishing enterprise
fundamentally rests on the strength of the peer review process. The journal
editor or the conference chair primarily relies on the expert reviewers'
assessment, identify points of agreement and disagreement and try to reach a
consensus to make a fair and informed decision on whether to accept or reject a
paper. However, with the escalating number of submissions requiring review,
especially in top-tier Artificial Intelligence (AI) conferences, the
editor/chair, among many other works, invests a significant, sometimes
stressful effort to mitigate reviewer disagreements. Here in this work, we
introduce a novel task of automatically identifying contradictions among
reviewers on a given article. To this end, we introduce ContraSciView, a
comprehensive review-pair contradiction dataset on around 8.5k papers (with
around 28k review pairs containing nearly 50k review pair comments) from the
open review-based ICLR and NeurIPS conferences. We further propose a baseline
model that detects contradictory statements from the review pairs. To the best
of our knowledge, we make the first attempt to identify disagreements among
peer reviewers automatically. We make our dataset and code public for further
investigations.

自动识别同行评审者之间的矛盾的研究，介绍了一个新颖的任务，通过对评论对之间的矛盾进行检测，为此我们提出了一个基准模型，并公开了数据集和代码供进一步研究。

科学同行评审中的评审者分歧发现

When Reviewers Lock Horn: Finding Disagreement in Scientific Peer  Reviews

Peer reviewing is a central component in the scientific publishing process.
We present the first public dataset of scientific peer reviews available for
research purposes (PeerRead v1) providing an opportunity to study this
important artifact. The dataset consists of 14.7K paper drafts and the
corresponding accept/reject decisions in top-tier venues including ACL, NIPS
and ICLR. The dataset also includes 10.7K textual peer reviews written by
experts for a subset of the papers. We describe the data collection process and
report interesting observed phenomena in the peer reviews. We also propose two
novel NLP tasks based on this dataset and provide simple baseline models. In
the first task, we show that simple models can predict whether a paper is
accepted with up to 21% error reduction compared to the majority baseline. In
the second task, we predict the numerical scores of review aspects and show
that simple models can outperform the mean baseline for aspects with high
variance such as 'originality' and 'impact'.

本文介绍了一个公开的科学领域数据集，提供了许多研究机会，该数据集集中在同行评审中使用的文本数据，此外，作者提出了两个基于 NLP 技术的新任务，包括简单的基准模型，第一个任务中，简单模型可以预测文章是否被接受，并且还可以在第二个任务中预测审稿方面的数值分数。