We propose Context Diffusion, a diffusion-based framework that enables image
generation models to learn from visual examples presented in context. Recent
work tackles such in-context learning for image generation, where a query image
is provided alongside context examples and text prompts. However, the quality
and fidelity of the generated images deteriorate when the prompt is not
present, demonstrating that these models are unable to truly learn from the
visual context. To address this, we propose a novel framework that separates
the encoding of the visual context and preserving the structure of the query
images. This results in the ability to learn from the visual context and text
prompts, but also from either one of them. Furthermore, we enable our model to
handle few-shot settings, to effectively address diverse in-context learning
scenarios. Our experiments and user study demonstrate that Context Diffusion
excels in both in-domain and out-of-domain tasks, resulting in an overall
enhancement in image quality and fidelity compared to counterpart models.

我们提出了 Context Diffusion，这是一种基于扩散的框架，使图像生成模型能够从环境中呈现的可视示例中学习。通过为查询图像提供上下文示例和文本提示，最近的工作解决了图像生成中的这种上下文学习问题。然而，当没有提示时，生成的图像的质量和保真度会降低，表明这些模型无法真正从视觉环境中学习。为了解决这个问题，我们提出了一种新颖的框架，将可视环境的编码和查询图像结构分开。这使得我们的模型能够从视觉环境和文本提示中学习，也能从其中任何一个中学习。此外，我们使我们的模型能够处理少量示例的设置，以有效地解决不同的上下文学习场景。我们的实验证明和用户研究表明，与对应的模型相比，Context Diffusion 在领域内和领域外的任务中都表现出色，从而提高了图像质量和保真度。