Diffusion models excel at creating visually-convincing images, but they often
struggle to meet subtle constraints inherent in the training data. Such
constraints could be physics-based (e.g., satisfying a PDE), geometric (e.g.,
respecting symmetry), or semantic (e.g., including a particular number of
objects). When the training data all satisfy a certain constraint, enforcing
this constraint on a diffusion model not only improves its
distribution-matching accuracy but also makes it more reliable for generating
valid synthetic data and solving constrained inverse problems. However,
existing methods for constrained diffusion models are inflexible with different
types of constraints. Recent work proposed to learn mirror diffusion models
(MDMs) in an unconstrained space defined by a mirror map and to impose the
constraint with an inverse mirror map, but analytical mirror maps are
challenging to derive for complex constraints. We propose neural approximate
mirror maps (NAMMs) for general constraints. Our approach only requires a
differentiable distance function from the constraint set. We learn an
approximate mirror map that pushes data into an unconstrained space and a
corresponding approximate inverse that maps data back to the constraint set. A
generative model, such as an MDM, can then be trained in the learned mirror
space and its samples restored to the constraint set by the inverse map. We
validate our approach on a variety of constraints, showing that compared to an
unconstrained diffusion model, a NAMM-based MDM substantially improves
constraint satisfaction. We also demonstrate how existing diffusion-based
inverse-problem solvers can be easily applied in the learned mirror space to
solve constrained inverse problems.

扩散模型与约束、镜像扩散模型、神经近似镜像映射和约束逆问题相关，通过学习镜像空间和映射方法改善扩散模型的约束满足性和解决约束逆问题的可靠性。

约束扩散模型的神经近似镜像地图

Neural Approximate Mirror Maps for Constrained Diffusion Models

Modern successes of diffusion models in learning complex, high-dimensional
data distributions are attributed, in part, to their capability to construct
diffusion processes with analytic transition kernels and score functions. The
tractability results in a simulation-free framework with stable regression
losses, from which reversed, generative processes can be learned at scale.
However, when data is confined to a constrained set as opposed to a standard
Euclidean space, these desirable characteristics appear to be lost based on
prior attempts. In this work, we propose Mirror Diffusion Models (MDM), a new
class of diffusion models that generate data on convex constrained sets without
losing any tractability. This is achieved by learning diffusion processes in a
dual space constructed from a mirror map, which, crucially, is a standard
Euclidean space. We derive efficient computation of mirror maps for popular
constrained sets, such as simplices and $\ell_2$-balls, showing significantly
improved performance of MDM over existing methods. For safety and privacy
purposes, we also explore constrained sets as a new mechanism to embed
invisible but quantitative information (i.e., watermarks) in generated data,
for which MDM serves as a compelling approach. Our work brings new algorithmic
opportunities for learning tractable diffusion on complex domains.

Mirror Diffusion Models 是一种新的扩散模型，能在凸约束集上生成数据而不损失可解性，并且通过学习镜像映射构造的对偶空间中的扩散过程，在应用于简单 xes 和∥2 - 球等流行约束集时，MDM 展现出显著改善的性能。此外，我们还探索了约束集作为一种新机制，将隐藏但定量信息（即水印）嵌入生成的数据中，以达到安全和隐私保护的目的。我们的工作为在复杂领域中学习可解扩散提供了新的算法机会。

镜像扩散模型用于受限和带水印的生成

Mirror Diffusion Models for Constrained and Watermarked Generation

Diffusion models have successfully been applied to generative tasks in
various continuous domains. However, applying diffusion to discrete categorical
data remains a non-trivial task. Moreover, generation in continuous domains
often requires clipping in practice, which motivates the need for a theoretical
framework for adapting diffusion to constrained domains. Inspired by the mirror
Langevin algorithm for the constrained sampling problem, in this theoretical
report we propose Mirror Diffusion Models (MDMs). We demonstrate MDMs in the
context of simplex diffusion and propose natural extensions to popular domains
such as image and text generation.

通过镜像扩散模型 (Mirror Diffusion Models, MDMs) 向离散分类数据应用镜像 Langevin 算法，本文提出了对连续域进行解释的一种理论框架，并在图像和文本生成等领域进行了实证研究。