Diffusion models have the ability to generate high quality images by
denoising pure Gaussian noise images. While previous research has primarily
focused on improving the control of image generation through adjusting the
denoising process, we propose a novel direction of manipulating the initial
noise to control the generated image. Through experiments on stable diffusion,
we show that blocks of pixels in the initial latent images have a preference
for generating specific content, and that modifying these blocks can
significantly influence the generated image. In particular, we show that
modifying a part of the initial image affects the corresponding region of the
generated image while leaving other regions unaffected, which is useful for
repainting tasks. Furthermore, we find that the generation preferences of pixel
blocks are primarily determined by their values, rather than their position. By
moving pixel blocks with a tendency to generate user-desired content to
user-specified regions, our approach achieves state-of-the-art performance in
layout-to-image generation. Our results highlight the flexibility and power of
initial image manipulation in controlling the generated image.

通过稳定的扩散实验，我们发现初始图像中的像素块倾向于生成特定的内容，我们通过修改它们可以显著影响生成的图像，这种影响只影响特定的区域，而不影响其他区域，这对于重绘任务非常有用。我们发现像素块的生成偏好主要由其值而不是位置决定，通过将具有生成用户所需内容倾向的像素块移动到用户指定的区域，我们的方法在布局到图像生成方面实现了最先进的性能，我们的结果凸显了初始图像操作在控制生成图像方面的灵活性和能力。

扩散模型中通过初始图像编辑实现引导图像合成

Guided Image Synthesis via Initial Image Editing in Diffusion Model

We consider grounding open domain dialogues with images. Existing work
assumes that both an image and a textual context are available, but
image-grounded dialogues by nature are more difficult to obtain than textual
dialogues. Thus, we propose learning a response generation model with both
image-grounded dialogues and textual dialogues by assuming that the visual
scene information at the time of a conversation can be represented by an image,
and trying to recover the latent images of the textual dialogues through
text-to-image generation techniques. The likelihood of the two types of
dialogues is then formulated by a response generator and an image reconstructor
that are learned within a conditional variational auto-encoding framework.
Empirical studies are conducted in both image-grounded conversation and
text-based conversation. In the first scenario, image-grounded dialogues,
especially under a low-resource setting, can be effectively augmented by
textual dialogues with latent images; while in the second scenario, latent
images can enrich the content of responses and at the same time keep them
relevant to contexts.

本文提出使用图像来增强样本数量的方法，以学习针对含有图像信息的对话的应答生成模型，并利用条件变分自编码框架下的响应生成器和图像重构器来刻画图像和文本信息之间的关系。实验表明，该方法在低资源情况下可以有效增加图像驱动类对话的训练样本，同时保持应答内容与上下文的相关性。

具有隐变量图像的开放域对话生成

Open Domain Dialogue Generation with Latent Images

A broad class of problems at the core of computational imaging, sensing, and
low-level computer vision reduces to the inverse problem of extracting latent
images that follow a prior distribution, from measurements taken under a known
physical image formation model. Traditionally, hand-crafted priors along with
iterative optimization methods have been used to solve such problems. In this
paper we present unrolled optimization with deep priors, a principled framework
for infusing knowledge of the image formation into deep networks that solve
inverse problems in imaging, inspired by classical iterative methods. We show
that instances of the framework outperform the state-of-the-art by a
substantial margin for a wide variety of imaging problems, such as denoising,
deblurring, and compressed sensing magnetic resonance imaging (MRI). Moreover,
we conduct experiments that explain how the framework is best used and why it
outperforms previous methods.

本文提出了一种使用深度先验知识进行解决图像重建问题的框架，该框架在去噪、去模糊以及磁共振成像的压缩感知等多种成像问题上实现了较大幅度的性能提升，并通过实验进一步解释了其优越性。