In this paper, we study the denoising diffusion probabilistic model (DDPM) in
wavelet space, instead of pixel space, for visual synthesis. Considering the
wavelet transform represents the image in spatial and frequency domains, we
carefully design a novel architecture SFUNet to effectively capture the
correlation for both domains. Specifically, in the standard denoising U-Net for
pixel data, we supplement the 2D convolutions and spatial-only attention layers
with our spatial frequency-aware convolution and attention modules to jointly
model the complementary information from spatial and frequency domains in
wavelet data. Our new architecture can be used as a drop-in replacement to the
pixel-based network and is compatible with the vanilla DDPM training process.
By explicitly modeling the wavelet signals, we find our model is able to
generate images with higher quality on CIFAR-10, FFHQ, LSUN-Bedroom, and
LSUN-Church datasets, than the pixel-based counterpart.

本研究使用 SFUNet 架构，以小波域而非像素空间中的 denoising diffusion probabilistic model (DDPM) 进行图像合成，通过捕捉空频域的相关性来改进像素数据的标准去噪 U-Net 结构，从而提高 CIFAR-10、FFHQ、LSUN-Bedroom 和 LSUN-Church 数据集上的图像合成质量。