Systematic review (SR) is a popular research method in software engineering
(SE). However, conducting an SR takes an average of 67 weeks. Thus, automating
any step of the SR process could reduce the effort associated with SRs. Our
objective is to investigate if Large Language Models (LLMs) can accelerate
title-abstract screening by simplifying abstracts for human screeners, and
automating title-abstract screening. We performed an experiment where humans
screened titles and abstracts for 20 papers with both original and simplified
abstracts from a prior SR. The experiment with human screeners was reproduced
with GPT-3.5 and GPT-4 LLMs to perform the same screening tasks. We also
studied if different prompting techniques (Zero-shot (ZS), One-shot (OS),
Few-shot (FS), and Few-shot with Chain-of-Thought (FS-CoT)) improve the
screening performance of LLMs. Lastly, we studied if redesigning the prompt
used in the LLM reproduction of screening leads to improved performance. Text
simplification did not increase the screeners' screening performance, but
reduced the time used in screening. Screeners' scientific literacy skills and
researcher status predict screening performance. Some LLM and prompt
combinations perform as well as human screeners in the screening tasks. Our
results indicate that the GPT-4 LLM is better than its predecessor, GPT-3.5.
Additionally, Few-shot and One-shot prompting outperforms Zero-shot prompting.
Using LLMs for text simplification in the screening process does not
significantly improve human performance. Using LLMs to automate title-abstract
screening seems promising, but current LLMs are not significantly more accurate
than human screeners. To recommend the use of LLMs in the screening process of
SRs, more research is needed. We recommend future SR studies publish
replication packages with screening data to enable more conclusive
experimenting with LLM screening.

系统性综述（SR）是软件工程（SE）中一种常见的研究方法。本研究旨在调查大型语言模型（LLMs）是否能通过简化摘要来加速标题 - 摘要筛选，并自动化标题 - 摘要筛选。研究结果表明，使用 LLMs 进行文本简化在筛选过程中并未显著提高人类的表现，但减少了筛选所需的时间。虽然当前的 LLMs 在筛选任务中并不比人类筛选者更准确，但使用 LLMs 自动化标题 - 摘要筛选似乎是有前景的。需要进一步的研究才能推荐在 SR 筛选过程中使用 LLMs。建议未来的 SR 研究发布包含筛选数据的复制包，以便进行更有说服力的 LLM 筛选实验。