Scientific writing is a challenging task, particularly for novice researchers
who often rely on feedback from experienced peers. Recent work has primarily
focused on improving surface form and style rather than manuscript content. In
this paper, we propose a novel task: automated focused feedback generation for
scientific writing assistance. We present SWIF$^{2}$T: a Scientific WrIting
Focused Feedback Tool. It is designed to generate specific, actionable and
coherent comments, which identify weaknesses in a scientific paper and/or
propose revisions to it. Our approach consists of four components - planner,
investigator, reviewer and controller - leveraging multiple Large Language
Models (LLMs) to implement them. We compile a dataset of 300 peer reviews
citing weaknesses in scientific papers and conduct human evaluation. The
results demonstrate the superiority in specificity, reading comprehension, and
overall helpfulness of SWIF$^{2}$T's feedback compared to other approaches. In
our analysis, we also identified cases where automatically generated reviews
were judged better than human ones, suggesting opportunities for integration of
AI-generated feedback in scientific writing.

科学写作反馈生成自动化辅助工具 SWIF$^{2}$T，通过使用多个大型语言模型实现四个组件：计划者、调查员、评论员和控制器，生成具体、可行和连贯的反馈，优于其他方法，并提出将 AI 生成的反馈整合到科学写作中的机会。

科学写作辅助的自动集中反馈生成

Automated Focused Feedback Generation for Scientific Writing Assistance

Complementing natural language (NL) requirements with graphical models can
improve stakeholders' communication and provide directions for system design.
However, creating models from requirements involves manual effort. The advent
of generative large language models (LLMs), ChatGPT being a notable example,
offers promising avenues for automated assistance in model generation. This
paper investigates the capability of ChatGPT to generate a specific type of
model, i.e., UML sequence diagrams, from NL requirements. We conduct a
qualitative study in which we examine the sequence diagrams generated by
ChatGPT for 28 requirements documents of various types and from different
domains. Observations from the analysis of the generated diagrams have
systematically been captured through evaluation logs, and categorized through
thematic analysis. Our results indicate that, although the models generally
conform to the standard and exhibit a reasonable level of understandability,
their completeness and correctness with respect to the specified requirements
often present challenges. This issue is particularly pronounced in the presence
of requirements smells, such as ambiguity and inconsistency. The insights
derived from this study can influence the practical utilization of LLMs in the
RE process, and open the door to novel RE-specific prompting strategies
targeting effective model generation.

ChatGPT 能够从自然语言需求中生成 UML 序列图，但生成的模型在完整性和正确性方面存在挑战，尤其在需求不明确和不一致的情况下。此研究对要求工程领域中的大型语言模型的实际应用和有效的模型生成提示策略具有重要影响。