Writing with next-phrase suggestions powered by large language models is
becoming more pervasive by the day. However, research to understand writers'
interaction and decision-making processes while engaging with such systems is
still emerging. We conducted a qualitative study to shed light on writers'
cognitive processes while writing with next-phrase suggestion systems. To do
so, we recruited 14 amateur writers to write two reviews each, one without
suggestions and one with suggestions. Additionally, we also positively and
negatively biased the suggestion system to get a diverse range of instances
where writers' opinions and the bias in the language model align or misalign to
varying degrees. We found that writers interact with next-phrase suggestions in
various complex ways: Writers abstracted and extracted multiple parts of the
suggestions and incorporated them within their writing, even when they
disagreed with the suggestion as a whole; along with evaluating the suggestions
on various criteria. The suggestion system also had various effects on the
writing process, such as altering the writer's usual writing plans, leading to
higher levels of distraction etc. Based on our qualitative analysis using the
cognitive process model of writing by Hayes as a lens, we propose a theoretical
model of 'writer-suggestion interaction' for writing with GPT-2 (and causal
language models in general) for a movie review writing task, followed by
directions for future research and design.

本研究通过定性分析，使用 Hayes 的认知过程模型研究了作者与 next-phrase suggestion 系统交互时的认知过程，得到了 “writer-suggestion interaction” 理论模型，为使用 GPT-2 和因果语言模型进行电影评论写作任务提供了方向。