Image editing affords increased control over the aesthetics and content of
generated images. Pre-existing works focus predominantly on text-based
instructions to achieve desired image modifications, which limit edit precision
and accuracy. In this work, we propose an inference-time editing optimisation,
designed to extend beyond textual edits to accommodate multiple editing
instruction types (e.g. spatial layout-based; pose, scribbles, edge maps). We
propose to disentangle the editing task into two competing subtasks: successful
local image modifications and global content consistency preservation, where
subtasks are guided through two dedicated loss functions. By allowing to adjust
the influence of each loss function, we build a flexible editing solution that
can be adjusted to user preferences. We evaluate our method using text, pose
and scribble edit conditions, and highlight our ability to achieve complex
edits, through both qualitative and quantitative experiments.

我们提出了一种推理时间编辑优化方法，旨在通过两个专门的损失函数，成功地完成本地图像修改和全局内容一致性保留的竞争子任务，实现文本、姿势和涂鸦等多种编辑指令类型的灵活编辑解决方案，并通过定性和定量实验证明了我们能够实现复杂的编辑。

基于优化的多模态语义图像编辑

Optimisation-Based Multi-Modal Semantic Image Editing

A prevailing paradigm in neural text generation is one-shot generation, where
text is produced in a single step. The one-shot setting is inadequate, however,
when the constraints the user wishes to impose on the generated text are
dynamic, especially when authoring longer documents. We address this limitation
with an interactive text generation setting in which the user interacts with
the system by issuing commands to edit existing text. To this end, we propose a
novel text editing task, and introduce WikiDocEdits, a dataset of
single-sentence edits crawled from Wikipedia. We show that our Interactive
Editor, a transformer-based model trained on this dataset, outperforms
baselines and obtains positive results in both automatic and human evaluations.
We present empirical and qualitative analyses of this model's performance.

本研究通过提出一种交互式文本生成设置，在其中用户通过向系统发出编辑现有文本的命令与系统进行交互，来解决神经文本生成中一次生成的局限性，并介绍了一种新的文本编辑任务。通过使用 Wikipedia 中的单句编辑构成了一个名为 WikiDocEdits 的数据集，使用基于 transformer 的模型在其中进行训练以提高其自动产生的成果和用户评估结果。在此基础上，本研究分别从经验和定性分析方面展示了该模型的性能表现。