Scientific facts are often spun in the popular press with the intent to
influence public opinion and action, as was evidenced during the COVID-19
pandemic. Automatic detection of misinformation in the scientific domain is
challenging because of the distinct styles of writing in these two media types
and is still in its nascence. Most research on the validity of scientific
reporting treats this problem as a claim verification challenge. In doing so,
significant expert human effort is required to generate appropriate claims. Our
solution bypasses this step and addresses a more real-world scenario where such
explicit, labeled claims may not be available. The central research question of
this paper is whether it is possible to use large language models (LLMs) to
detect misinformation in scientific reporting. To this end, we first present a
new labeled dataset SciNews, containing 2.4k scientific news stories drawn from
trusted and untrustworthy sources, paired with related abstracts from the
CORD-19 database. Our dataset includes both human-written and LLM-generated
news articles, making it more comprehensive in terms of capturing the growing
trend of using LLMs to generate popular press articles. Then, we identify
dimensions of scientific validity in science news articles and explore how this
can be integrated into the automated detection of scientific misinformation. We
propose several baseline architectures using LLMs to automatically detect false
representations of scientific findings in the popular press. For each of these
architectures, we use several prompt engineering strategies including
zero-shot, few-shot, and chain-of-thought prompting. We also test these
architectures and prompting strategies on GPT-3.5, GPT-4, and Llama2-7B,
Llama2-13B.

使用大型语言模型检测科学报道中的虚假信息，针对缺乏明确标签的情况，提出了多种基于大型语言模型的基准架构和提示方法，包括零样本、少样本和连贯思维提示等。

大型语言模型能否检测科学新闻报道中的错误信息？

Can Large Language Models Detect Misinformation in Scientific News  Reporting?

Anthropomorphism, or the attribution of human-like characteristics to
non-human entities, has shaped conversations about the impacts and
possibilities of technology. We present AnthroScore, an automatic metric of
implicit anthropomorphism in language. We use a masked language model to
quantify how non-human entities are implicitly framed as human by the
surrounding context. We show that AnthroScore corresponds with human judgments
of anthropomorphism and dimensions of anthropomorphism described in social
science literature. Motivated by concerns of misleading anthropomorphism in
computer science discourse, we use AnthroScore to analyze 15 years of research
papers and downstream news articles. In research papers, we find that
anthropomorphism has steadily increased over time, and that papers related to
language models have the most anthropomorphism. Within ACL papers, temporal
increases in anthropomorphism are correlated with key neural advancements.
Building upon concerns of scientific misinformation in mass media, we identify
higher levels of anthropomorphism in news headlines compared to the research
papers they cite. Since AnthroScore is lexicon-free, it can be directly applied
to a wide range of text sources.

利用自然语言处理技术，作者提出了 AnthroScore，一种用于测量文本中隐含人性化的自动化指标，该指标与人类对人性化的判断和社会科学文献中所描述的人性化维度相关。作者的研究表明，随着时间的推移，人性化在研究论文中逐渐增加，并且与语言模型相关的论文具有最高的人性化程度。此外，作者还发现新闻标题中的人性化水平较高，并与其引用的研究论文相比存在科学误导。AnthroScore 无需词汇表即可直接应用于各种文本来源。