Advances in language modeling have paved the way for novel human-AI
co-writing experiences. This paper explores how varying levels of scaffolding
from large language models (LLMs) shape the co-writing process. Employing a
within-subjects field experiment with a Latin square design, we asked
participants (N=131) to respond to argumentative writing prompts under three
randomly sequenced conditions: no AI assistance (control), next-sentence
suggestions (low scaffolding), and next-paragraph suggestions (high
scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing
quality and productivity (words/time). While low scaffolding did not
significantly improve writing quality or productivity, high scaffolding led to
significant improvements, especially benefiting non-regular writers and less
tech-savvy users. No significant cognitive burden was observed while using the
scaffolded writing tools, but a moderate decrease in text ownership and
satisfaction was noted. Our results have broad implications for the design of
AI-powered writing tools, including the need for personalized scaffolding
mechanisms.

大语言模型的不同程度脚手架支持对协作写作过程的影响进行了探讨。在一个拉丁方设计的实地实验中，共有 131 位参与者在无 AI 辅助（对照组）、下一句建议（低脚手架）和下一段建议（高脚手架）三种随机顺序的条件下回答辩论写作问题。研究发现，脚手架对写作质量和生产率（单位时间内的字数）的影响呈 U 型曲线。低脚手架并没有显著改善写作质量或生产率，而高脚手架则显著改善，特别是对于非常规写作者和对技术不熟悉的用户有益。使用脚手架写作工具时没有观察到明显的认知负荷，但注意到了对文本所有权和满意度的适度减少。我们的研究对于设计 AI 辅助写作工具具有广泛的意义，包括个性化脚手架机制的需求。

塑造人工智能协作：与语言模型共同撰写中的多层支架水平

Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing  with Language Models

Recent developments in natural language generation (NLG) using neural
language models have brought us closer than ever to the goal of building
AI-powered creative writing tools. However, most prior work on human-AI
collaboration in the creative writing domain has evaluated new systems with
amateur writers, typically in contrived user studies of limited scope. In this
work, we commissioned 13 professional, published writers from a diverse set of
creative writing backgrounds to craft stories using Wordcraft, a text editor
with built-in AI-powered writing assistance tools. Using interviews and
participant journals, we discuss the potential of NLG to have significant
impact in the creative writing domain--especially with respect to
brainstorming, generation of story details, world-building, and research
assistance. Experienced writers, more so than amateurs, typically have
well-developed systems and methodologies for writing, as well as distinctive
voices and target audiences. Our work highlights the challenges in building for
these writers; NLG technologies struggle to preserve style and authorial voice,
and they lack deep understanding of story contents. In order for AI-powered
writing assistants to realize their full potential, it is essential that they
take into account the diverse goals and expertise of human writers.

本文研究了使用 AI 技术辅助创作的自然语言生成（NLG）模型，并邀请了 13 位专业作家来尝试使用 Wordcraft，这个内置 AI 辅助撰写工具的文本编辑器。结果显示 NLG 工具在创作过程的脑暴、描写细节、建立故事世界和研究文本方面具有很大的潜力。但由于 NLG 技术在保留风格、语调和理解故事内容等方面存在困难，因此建立一个合适的作家个性化 NLG 辅助工具是非常有挑战性的。