Diffusion-based image generation models have achieved great success in recent
years by showing the capability of synthesizing high-quality content. However,
these models contain a huge number of parameters, resulting in a significantly
large model size. Saving and transferring them is a major bottleneck for
various applications, especially those running on resource-constrained devices.
In this work, we develop a novel weight quantization method that quantizes the
UNet from Stable Diffusion v1.5 to 1.99 bits, achieving a model with 7.9X
smaller size while exhibiting even better generation quality than the original
one. Our approach includes several novel techniques, such as assigning optimal
bits to each layer, initializing the quantized model for better performance,
and improving the training strategy to dramatically reduce quantization error.
Furthermore, we extensively evaluate our quantized model across various
benchmark datasets and through human evaluation to demonstrate its superior
generation quality.

通过开发一种新的权重量化方法，将稳定扩散 v1.5 的 UNet 模型量化为 1.99 位，实现模型尺寸减小 7.9 倍，同时展现更好的生成质量。

BitsFusion：扩散模型的 1.99 位权重量化

BitsFusion: 1.99 bits Weight Quantization of Diffusion Model

This paper presents a novel method for exerting fine-grained lighting control
during text-driven diffusion-based image generation. While existing diffusion
models already have the ability to generate images under any lighting
condition, without additional guidance these models tend to correlate image
content and lighting. Moreover, text prompts lack the necessary expressional
power to describe detailed lighting setups. To provide the content creator with
fine-grained control over the lighting during image generation, we augment the
text-prompt with detailed lighting information in the form of radiance hints,
i.e., visualizations of the scene geometry with a homogeneous canonical
material under the target lighting. However, the scene geometry needed to
produce the radiance hints is unknown. Our key observation is that we only need
to guide the diffusion process, hence exact radiance hints are not necessary;
we only need to point the diffusion model in the right direction. Based on this
observation, we introduce a three stage method for controlling the lighting
during image generation. In the first stage, we leverage a standard pretrained
diffusion model to generate a provisional image under uncontrolled lighting.
Next, in the second stage, we resynthesize and refine the foreground object in
the generated image by passing the target lighting to a refined diffusion
model, named DiLightNet, using radiance hints computed on a coarse shape of the
foreground object inferred from the provisional image. To retain the texture
details, we multiply the radiance hints with a neural encoding of the
provisional synthesized image before passing it to DiLightNet. Finally, in the
third stage, we resynthesize the background to be consistent with the lighting
on the foreground object. We demonstrate and validate our lighting controlled
diffusion model on a variety of text prompts and lighting conditions.

该论文提出了一种新的方法来在基于扩散的图像生成过程中实现细粒度的灯光控制，通过为文本提示添加辐射提示的详细光照信息，结合经过预训练的扩散模型和 DiLightNet 来控制图像生成过程中的灯光。

DiLightNet：面向基于扩散的图像生成的细粒度照明控制

DiLightNet: Fine-grained Lighting Control for Diffusion-based Image  Generation

In recent years, machine learning models like DALL-E, Craiyon, and Stable
Diffusion have gained significant attention for their ability to generate
high-resolution images from concise descriptions. Concurrently, quantum
computing is showing promising advances, especially with quantum machine
learning which capitalizes on quantum mechanics to meet the increasing
computational requirements of traditional machine learning algorithms. This
paper explores the integration of quantum machine learning and variational
quantum circuits to augment the efficacy of diffusion-based image generation
models. Specifically, we address two challenges of classical diffusion models:
their low sampling speed and the extensive parameter requirements. We introduce
two quantum diffusion models and benchmark their capabilities against their
classical counterparts using MNIST digits, Fashion MNIST, and CIFAR-10. Our
models surpass the classical models with similar parameter counts in terms of
performance metrics FID, SSIM, and PSNR. Moreover, we introduce a consistency
model unitary single sampling architecture that combines the diffusion
procedure into a single step, enabling a fast one-step image generation.

本研究探讨了量子机器学习和变分量子电路的整合，以增强扩散式图像生成模型的效力，并通过 MNIST 数字，Fashion MNIST 和 CIFAR-10 对两种量子扩散模型进行了基准测试，结果显示我们的模型在性能指标 FID、SSIM 和 PSNR 方面超越了传统模型。此外，我们还引入了一种一致性模型单一采样架构，将扩散过程合并为一步，实现了快速一步图像生成。

量子去噪扩散模型

Quantum Denoising Diffusion Models

Recently, diffusion-based image generation methods are credited for their
remarkable text-to-image generation capabilities, while still facing challenges
in accurately generating multilingual scene text images. To tackle this
problem, we propose Diff-Text, which is a training-free scene text generation
framework for any language. Our model outputs a photo-realistic image given a
text of any language along with a textual description of a scene. The model
leverages rendered sketch images as priors, thus arousing the potential
multilingual-generation ability of the pre-trained Stable Diffusion. Based on
the observation from the influence of the cross-attention map on object
placement in generated images, we propose a localized attention constraint into
the cross-attention layer to address the unreasonable positioning problem of
scene text. Additionally, we introduce contrastive image-level prompts to
further refine the position of the textual region and achieve more accurate
scene text generation. Experiments demonstrate that our method outperforms the
existing method in both the accuracy of text recognition and the naturalness of
foreground-background blending.

提出了 Diff-Text，它是一个训练免费的场景文本生成框架，能够以任何语言的文本和场景的文本描述为输入，输出逼真的照片。通过引入局部化的注意力约束和对比度图像级提示，实现了场景文本的准确生成，并在文本识别准确性和前景背景融合的自然度方面优于现有方法。