Diffusion Probabilistic Models (DPMs) have shown a powerful capacity of
generating high-quality image samples. Recently, diffusion autoencoders
(Diff-AE) have been proposed to explore DPMs for representation learning via
autoencoding. Their key idea is to jointly train an encoder for discovering
meaningful representations from images and a conditional DPM as the decoder for
reconstructing images. Considering that training DPMs from scratch will take a
long time and there have existed numerous pre-trained DPMs, we propose
\textbf{P}re-trained \textbf{D}PM \textbf{A}uto\textbf{E}ncoding
(\textbf{PDAE}), a general method to adapt existing pre-trained DPMs to the
decoders for image reconstruction, with better training efficiency and
performance than Diff-AE. Specifically, we find that the reason that
pre-trained DPMs fail to reconstruct an image from its latent variables is due
to the information loss of forward process, which causes a gap between their
predicted posterior mean and the true one. From this perspective, the
classifier-guided sampling method can be explained as computing an extra mean
shift to fill the gap, reconstructing the lost class information in samples.
These imply that the gap corresponds to the lost information of the image, and
we can reconstruct the image by filling the gap. Drawing inspiration from this,
we employ a trainable model to predict a mean shift according to encoded
representation and train it to fill as much gap as possible, in this way, the
encoder is forced to learn as much information as possible from images to help
the filling. By reusing a part of network of pre-trained DPMs and redesigning
the weighting scheme of diffusion loss, PDAE can learn meaningful
representations from images efficiently. Extensive experiments demonstrate the
effectiveness, efficiency and flexibility of PDAE.

本文提出了一种基于现有预训练 DPM 模型的图像重建通用方法 PDAE，实现了比 Diff-AE 更好的训练效率和性能，并通过对分类器引导采样的机制填补信息缺失的空隙，使编码器对图像进行更好的重建，表现出了较高的有效性、效率和灵活性。