Maximum likelihood (ML) learning for energy-based models (EBMs) is
challenging, partly due to non-convergence of Markov chain Monte Carlo.Several
variations of ML learning have been proposed, but existing methods all fail to
achieve both post-training image generation and proper density estimation. We
propose to introduce diffusion data and learn a joint EBM, called diffusion
assisted-EBMs, through persistent training (i.e., using persistent contrastive
divergence) with an enhanced sampling algorithm to properly sample from
complex, multimodal distributions. We present results from a 2D illustrative
experiment and image experiments and demonstrate that, for the first time for
image data, persistently trained EBMs can {\it simultaneously} achieve long-run
stability, post-training image generation, and superior out-of-distribution
detection.

本文介绍了通过引入扩散数据和使用增强采样算法，通过持续对比散度学习一种联合的能量基模型，称为扩散辅助 - EBM，来实现复杂的，多模态分布的准确采样和密度估计，并进行了 2D 和图像实验，证明了持续训练的 EBMs 可以同时实现长时间稳定性，后训练图像生成和卓越的异常检测。

持续训练的扩散辅助能量模型

Persistently Trained, Diffusion-assisted Energy-based Models

Learning nonlinear dynamics from diffusion data is a challenging problem
since the individuals observed may be different at different time points,
generally following an aggregate behaviour. Existing work cannot handle the
tasks well since they model such dynamics either directly on observations or
enforce the availability of complete longitudinal individual-level
trajectories. However, in most of the practical applications, these
requirements are unrealistic: the evolving dynamics may be too complex to be
modeled directly on observations, and individual-level trajectories may not be
available due to technical limitations, experimental costs and/or privacy
issues. To address these challenges, we formulate a model of diffusion dynamics
as the {\em hidden stochastic process} via the introduction of hidden variables
for flexibility, and learn the hidden dynamics directly on {\em aggregate
observations} without any requirement for individual-level trajectories. We
propose a dynamic generative model with Wasserstein distance for LEarninG dEep
hidden Nonlinear Dynamics (LEGEND) and prove its theoretical guarantees as
well. Experiments on a range of synthetic and real-world datasets illustrate
that LEGEND has very strong performance compared to state-of-the-art baselines.

通过引入具有灵活性的隐藏变量，使用 Wasserstein distance 的动态生成模型可以从聚合观察数据中直接学习非线性动力学，并且相对于现有技术具有很强的性能表现。