Generative Models (GMs) have attracted considerable attention due to their
tremendous success in various domains, such as computer vision where they are
capable to generate impressive realistic-looking images. Likelihood-based GMs
are attractive due to the possibility to generate new data by a single model
evaluation. However, they typically achieve lower sample quality compared to
state-of-the-art score-based diffusion models (DMs). This paper provides a
significant step in the direction of addressing this limitation. The idea is to
borrow one of the strengths of score-based DMs, which is the ability to perform
accurate density estimation in low-density regions and to address manifold
overfitting by means of data mollification. We connect data mollification
through the addition of Gaussian noise to Gaussian homotopy, which is a
well-known technique to improve optimization. Data mollification can be
implemented by adding one line of code in the optimization loop, and we
demonstrate that this provides a boost in generation quality of
likelihood-based GMs, without computational overheads. We report results on
image data sets with popular likelihood-based GMs, including variants of
variational autoencoders and normalizing flows, showing large improvements in
FID score.

通过将 score-based diffusion models 中密度估计的方法应用到 likelihood-based generative models 中，通过添加高斯噪音实现 data mollification，提高生成画质，对多种基于 likelihood 的 GMs 进行了测试，并在 FID score 上取得了显著的提升。

通过一行代码数据平滑改善似然生成模型的优化

One-Line-of-Code Data Mollification Improves Optimization of  Likelihood-based Generative Models

Denoising diffusion (score-based) generative models have recently achieved
significant accomplishments in generating realistic and diverse data. These
approaches define a forward diffusion process for transforming data into noise
and a backward denoising process for sampling data from noise. Unfortunately,
the generation process of current denoising diffusion models is notoriously
slow due to the lengthy iterative noise estimations, which rely on cumbersome
neural networks. It prevents the diffusion models from being widely deployed,
especially on edge devices. Previous works accelerate the generation process of
diffusion model (DM) via finding shorter yet effective sampling trajectories.
However, they overlook the cost of noise estimation with a heavy network in
every iteration. In this work, we accelerate generation from the perspective of
compressing the noise estimation network. Due to the difficulty of retraining
DMs, we exclude mainstream training-aware compression paradigms and introduce
post-training quantization (PTQ) into DM acceleration. However, the output
distributions of noise estimation networks change with time-step, making
previous PTQ methods fail in DMs since they are designed for single-time step
scenarios. To devise a DM-specific PTQ method, we explore PTQ on DM in three
aspects: quantized operations, calibration dataset, and calibration metric. We
summarize and use several observations derived from all-inclusive
investigations to formulate our method, which especially targets the unique
multi-time-step structure of DMs. Experimentally, our method can directly
quantize full-precision DMs into 8-bit models while maintaining or even
improving their performance in a training-free manner. Importantly, our method
can serve as a plug-and-play module on other fast-sampling methods, e.g., DDIM.
The code is available at this https URL .

本文介绍了如何加速去噪扩散生成模型的生成过程，通过对去噪网络进行压缩，在不进行重新训练的情况下，将完全精度的 DM 量化为 8 位模型，并可在其他快速采样方法上使用。

扩散模型的训练后量化

Post-training Quantization on Diffusion Models

We consider the problem of learning a Bayesian network or directed acyclic
graph (DAG) model from observational data. A number of constraint-based,
score-based and hybrid algorithms have been developed for this purpose. For
constraint-based methods, statistical consistency guarantees typically rely on
the faithfulness assumption, which has been show to be restrictive especially
for graphs with cycles in the skeleton. However, there is only limited work on
consistency guarantees for score-based and hybrid algorithms and it has been
unclear whether consistency guarantees can be proven under weaker conditions
than the faithfulness assumption. In this paper, we propose the sparsest
permutation (SP) algorithm. This algorithm is based on finding the causal
ordering of the variables that yields the sparsest DAG. We prove that this new
score-based method is consistent under strictly weaker conditions than the
faithfulness assumption. We also demonstrate through simulations on small DAGs
that the SP algorithm compares favorably to the constraint-based PC and SGS
algorithms as well as the score-based Greedy Equivalence Search and hybrid
Max-Min Hill-Climbing method. In the Gaussian setting, we prove that our
algorithm boils down to finding the permutation of the variables with sparsest
Cholesky decomposition for the inverse covariance matrix. Using this
connection, we show that in the oracle setting, where the true covariance
matrix is known, the SP algorithm is in fact equivalent to $\ell_0$-penalized
maximum likelihood estimation.

该研究提出了一种新的基于分数的算法（SP 算法）来学习贝叶斯网络或有向无环图（DAG）模型，该算法在弱于忠实性假设的条件下具有一致性保证，并且在一些小 DAG 的模拟中表现出与其他算法相比的优越性。