The new polymath Large Language Models (LLMs) can speed-up greatly scientific
reviews, possibly using more unbiased quantitative metrics, facilitating
cross-disciplinary connections, and identifying emerging trends and research
gaps by analyzing large volumes of data. However, at the present time, they
lack the required deep understanding of complex methodologies, they have
difficulty in evaluating innovative claims, and they are unable to assess
ethical issues and conflicts of interest. Herein, we consider 13 GPT-related
papers across different scientific domains, reviewed by a human reviewer and
SciSpace, a large language model, with the reviews evaluated by three distinct
types of evaluators, namely GPT-3.5, a crowd panel, and GPT-4. We found that
50% of SciSpace's responses to objective questions align with those of a human
reviewer, with GPT-4 (informed evaluator) often rating the human reviewer
higher in accuracy, and SciSpace higher in structure, clarity, and
completeness. In subjective questions, the uninformed evaluators (GPT-3.5 and
crowd panel) showed varying preferences between SciSpace and human responses,
with the crowd panel showing a preference for the human responses. However,
GPT-4 rated them equally in accuracy and structure but favored SciSpace for
completeness.

新型多面手大语言模型可通过分析大量数据加快科学评审，使用更无偏的定量度量指标，促进跨学科的联系，确定新兴趋势和研究领域，并通过评估大量数据来识别的方法，但目前它们缺乏对复杂方法的深入理解，评估创新性的主张存在困难，并且无法评估伦理问题和利益冲突。