Instruction-based image editing improves the controllability and flexibility
of image manipulation via natural commands without elaborate descriptions or
regional masks. However, human instructions are sometimes too brief for current
methods to capture and follow. Multimodal large language models (MLLMs) show
promising capabilities in cross-modal understanding and visual-aware response
generation via LMs. We investigate how MLLMs facilitate edit instructions and
present MLLM-Guided Image Editing (MGIE). MGIE learns to derive expressive
instructions and provides explicit guidance. The editing model jointly captures
this visual imagination and performs manipulation through end-to-end training.
We evaluate various aspects of Photoshop-style modification, global photo
optimization, and local editing. Extensive experimental results demonstrate
that expressive instructions are crucial to instruction-based image editing,
and our MGIE can lead to a notable improvement in automatic metrics and human
evaluation while maintaining competitive inference efficiency.

通过多模态大语言模型（MLLMs）的指导，MLLM-Guided Image Editing（MGIE）学习能够提供表达性指令和明确引导的图像编辑模型，其通过端到端训练同时捕捉了视觉想象力并执行图像操作。大量实验结果证明，表达性指令对基于指令的图像编辑至关重要，而我们的 MGIE 在保持有竞争力的推理效率的同时，能够显著改善自动度量和人类评估。