Recently, images are considered samples from a high-dimensional distribution,
and deep learning has become almost synonymous with image generation. However,
is a deep learning network truly necessary for image generation? In this paper,
we investigate the possibility of image generation without using a deep
learning network, motivated by validating the assumption that images follow a
high-dimensional distribution. Since images are assumed to be samples from such
a distribution, we utilize the Gaussian Mixture Model (GMM) to describe it. In
particular, we employ a recent distribution learning technique named as
Monte-Carlo Marginalization to capture the parameters of the GMM based on image
samples. Moreover, we also use the Singular Value Decomposition (SVD) for
dimensionality reduction to decrease computational complexity. During our
evaluation experiment, we first attempt to model the distribution of image
samples directly to verify the assumption that images truly follow a
distribution. We then use the SVD for dimensionality reduction. The principal
components, rather than raw image data, are used for distribution learning.
Compared to methods relying on deep learning networks, our approach is more
explainable, and its performance is promising. Experiments show that our images
have a lower FID value compared to those generated by variational
auto-encoders, demonstrating the feasibility of image generation without deep
learning networks.

该研究通过验证图像遵循高维分布的假设，利用高斯混合模型（GMM）和 SVD 降维的方法，探讨了不使用深度学习网络进行图像生成的可能性，并表明相比于使用深度学习网络方法，该方法具有更强的可解释性和良好的性能。

深度学习网络在图像生成中是否必要？

Is Deep Learning Network Necessary for Image Generation?

We propose a novel method to learn intractable distributions from their
samples. The main idea is to use a parametric distribution model, such as a
Gaussian Mixture Model (GMM), to approximate intractable distributions by
minimizing the KL-divergence. Based on this idea, there are two challenges that
need to be addressed. First, the computational complexity of KL-divergence is
unacceptable when the dimensions of distributions increases. The Monte-Carlo
Marginalization (MCMarg) is proposed to address this issue. The second
challenge is the differentiability of the optimization process, since the
target distribution is intractable. We handle this problem by using Kernel
Density Estimation (KDE). The proposed approach is a powerful tool to learn
complex distributions and the entire process is differentiable. Thus, it can be
a better substitute of the variational inference in variational auto-encoders
(VAE). One strong evidence of the benefit of our method is that the
distributions learned by the proposed approach can generate better images even
based on a pre-trained VAE's decoder. Based on this point, we devise a
distribution learning auto-encoder which is better than VAE under the same
network architecture. Experiments on standard dataset and synthetic data
demonstrate the efficiency of the proposed approach.

提出一种从样本中学习难以处理的分布的新方法，通过最小化 KL 散度使用参数分布模型（如高斯混合模型）来近似不可解的分布。通过引入 Monte-Carlo 边缘化和 Kernel 密度估计来解决计算复杂性和非可导优化过程的挑战，该方法能够学习复杂分布并生成更好的图片。