By organizing knowledge within a research field, Systematic Reviews (SR) provide valuable leads to steer research. Evidence suggests that SRs have become first-class artifacts in software engineering. However, the tedious manual effort associated with the screening phase of SRs renders these studies a costly and error-prone endeavor. While screening has traditionally been considered not amenable to automation, the advent of generative AI-driven chatbots, backed with large language models is set to disrupt the field. In this report, we propose an approach to leverage these novel technological developments for automating the screening of SRs. We assess the consistency, classification performance, and generalizability of ChatGPT in screening articles for SRs and compare these figures with those of traditional classifiers used in SR automation. Our results indicate that ChatGPT is a viable option to automate the SR processes, but requires careful considerations from developers when integrating ChatGPT into their SR tools.

本报告提出一种利用最新技术发展自动化 Systematic Reviews 筛选的方法，通过与传统分类器相比较的一系列实验，证明了 ChatGPT 成为自动化 SR 过程的一种可行选择，但需要开发人员仔细考虑在 SR 工具中集成 ChatGPT 的情况。

评估 ChatGPT 在系统性综述文章筛选方面的能力