Scientific facts are often spun in the popular press with the intent to
influence public opinion and action, as was evidenced during the COVID-19
pandemic. Automatic detection of misinformation in the scientific domain is
challenging because of the distinct styles of writing in these two media types
and is still in its nascence. Most research on the validity of scientific
reporting treats this problem as a claim verification challenge. In doing so,
significant expert human effort is required to generate appropriate claims. Our
solution bypasses this step and addresses a more real-world scenario where such
explicit, labeled claims may not be available. The central research question of
this paper is whether it is possible to use large language models (LLMs) to
detect misinformation in scientific reporting. To this end, we first present a
new labeled dataset SciNews, containing 2.4k scientific news stories drawn from
trusted and untrustworthy sources, paired with related abstracts from the
CORD-19 database. Our dataset includes both human-written and LLM-generated
news articles, making it more comprehensive in terms of capturing the growing
trend of using LLMs to generate popular press articles. Then, we identify
dimensions of scientific validity in science news articles and explore how this
can be integrated into the automated detection of scientific misinformation. We
propose several baseline architectures using LLMs to automatically detect false
representations of scientific findings in the popular press. For each of these
architectures, we use several prompt engineering strategies including
zero-shot, few-shot, and chain-of-thought prompting. We also test these
architectures and prompting strategies on GPT-3.5, GPT-4, and Llama2-7B,
Llama2-13B.

使用大型语言模型检测科学报道中的虚假信息，针对缺乏明确标签的情况，提出了多种基于大型语言模型的基准架构和提示方法，包括零样本、少样本和连贯思维提示等。