Image editing is an iterative process that requires precise visual evaluation and manipulation for the output to match the editing intent. However, current image editing tools do not provide accessible interaction nor sufficient feedback for blind and low vision individuals to achieve this level of control. To address this, we developed EditScribe, a prototype system that makes image editing accessible using natural language verification loops powered by large multimodal models. Using EditScribe, the user first comprehends the image content through initial general and object descriptions, then specifies edit actions using open-ended natural language prompts. EditScribe performs the image edit, and provides four types of verification feedback for the user to verify the performed edit, including a summary of visual changes, AI judgement, and updated general and object descriptions. The user can ask follow-up questions to clarify and probe into the edits or verification feedback, before performing another edit. In a study with ten blind or low-vision users, we found that EditScribe supported participants to perform and verify image edit actions non-visually. We observed different prompting strategies from participants, and their perceptions on the various types of verification feedback. Finally, we discuss the implications of leveraging natural language verification loops to make visual authoring non-visually accessible.

本研究解决了盲人和低视力人士在图像编辑中面临的交互和反馈不足的问题。我们提出了一种名为EditScribe的原型系统，通过大型多模态模型，基于自然语言的验证循环，使图像编辑过程更具可访问性。研究发现，EditScribe能够支持用户以非视觉的方式执行和验证图像编辑操作，展示了自然语言验证循环在视觉创作中的潜在影响。

EditScribe: 基于自然语言验证循环的非视觉图像编辑