Evaluating the significance of a paper is pivotal yet challenging for the
scientific community. While the citation count is the most commonly used proxy
for this purpose, they are widely criticized for failing to accurately reflect
a paper's true impact. In this work, we propose a causal inference method,
TextMatch, which adapts the traditional matching framework to high-dimensional
text embeddings. Specifically, we encode each paper using the text embeddings
by large language models (LLMs), extract similar samples by cosine similarity,
and synthesize a counterfactual sample by the weighted average of similar
papers according to their similarity values. We apply the resulting metric,
called CausalCite, as a causal formulation of paper citations. We show its
effectiveness on various criteria, such as high correlation with paper impact
as reported by scientific experts on a previous dataset of 1K papers,
(test-of-time) awards for past papers, and its stability across various
sub-fields of AI. We also provide a set of findings that can serve as suggested
ways for future researchers to use our metric for a better understanding of a
paper's quality. Our code and data are at
this https URL

本研究提出了一种叫做 CausalCite 的因果推断方法，通过高维文本嵌入将每篇论文编码，利用余弦相似度提取相似的样本并综合计算出反事实样本，用于衡量论文被引用的因果关系，该方法在科学专家对 1,000 篇论文进行的实验、历史论文的奖项和 AI 各个子领域的稳定性等各种标准中表现出了很高的相关性和有效性，并给出了一些建议供未来研究者利用我们的度量方法以更好地理解论文质量。

CausalCite：论文引用的因果表述

CausalCite: A Causal Formulation of Paper Citations

In recent years, funding agencies and journals increasingly advocate for open
science practices (e.g. data and method sharing) to improve the transparency,
access, and reproducibility of science. However, quantifying these practices at
scale has proven difficult. In this work, we leverage a large-scale dataset of
1.1M papers from arXiv that are representative of the fields of physics, math,
and computer science to analyze the adoption of data and method link-sharing
practices over time and their impact on article reception. To identify links to
data and methods, we train a neural text classification model to automatically
classify URL types based on contextual mentions in papers. We find evidence
that the practice of link-sharing to methods and data is spreading as more
papers include such URLs over time. Reproducibility efforts may also be
spreading because the same links are being increasingly reused across papers
(especially in computer science); and these links are increasingly concentrated
within fewer web domains (e.g. Github) over time. Lastly, articles that share
data and method links receive increased recognition in terms of citation count,
with a stronger effect when the shared links are active (rather than defunct).
Together, these findings demonstrate the increased spread and perceived value
of data and method sharing practices in open science.

近年来，资助机构和期刊越来越倡导开放科学实践（如数据和方法共享），以提高科学的透明度、可获取性和可重复性。本研究利用来自 arXiv 的 110 万篇论文的大规模数据集，代表了物理、数学和计算机科学领域，分析了随时间推移数据和方法链接共享实践的采用情况以及对文章接受程度的影响。我们通过训练神经文本分类模型来识别数据和方法链接，根据论文中的上下文提及自动分类 URL 类型。研究发现，随着时间推移，链接方法和数据的共享实践正在扩散，越来越多的论文包含此类 URL。可重复性努力也可能在扩散，因为相同的链接越来越多地在论文中被重复使用（特别是计算机科学领域），并且这些链接随时间越来越集中在更少的网域（如 Github）中。最后，分享数据和方法链接的文章在引用次数方面获得了增加的承认，当链接是有效的时候效果更强。这些发现共同证明了数据和方法共享实践在开放科学中的扩展和受到的重视。

开放科学的兴起：追踪数据和方法共享实践的演变和感知价值

The Rise of Open Science: Tracking the Evolution and Perceived Value of  Data and Methods Link-Sharing Practices

I propose the index $\hbar$ ("hbar"), defined as the number of papers of an
individual that have citation count larger than or equal to the $\hbar$ of all
coauthors of each paper, as a useful index to characterize the scientific
output of a researcher that takes into account the effect of multiple
coauthorship. The bar is higher for $\hbar$.

提出了指数 hbar，用于表征研究人员的科学产出，其考虑了多人合著的影响，hbar 的值会因合著者的论文引用量而有所变化。