This paper studies a curious phenomenon in learning energy-based model (EBM)
using MCMC. In each learning iteration, we generate synthesized examples by
running a non-convergent, non-mixing, and non-persistent short-run MCMC toward
the current model, always starting from the same initial distribution such as
uniform noise distribution, and always running a fixed number of MCMC steps.
After generating synthesized examples, we then update the model parameters
according to the maximum likelihood learning gradient, as if the synthesized
examples are fair samples from the current model. We treat this non-convergent
short-run MCMC as a learned generator model or a flow model. We provide
arguments for treating the learned non-convergent short-run MCMC as a valid
model. We show that the learned short-run MCMC is capable of generating
realistic images. More interestingly, unlike traditional EBM or MCMC, the
learned short-run MCMC is capable of reconstructing observed images and
interpolating between images, like generator or flow models. The code can be
found in the Appendix.

本文研究了使用 MCMC 学习能量模型（EBM）中的一个有趣现象。我们将非收敛、不混合、不持久的短程 MCMC 作为学习非收敛器模型或流模型，在生成实例后，我们将按照最大似然学习梯度来更新模型参数，就好像合成实例是从当前模型中公平地抽样一样。此外，我们还展示了学习的短程 MCMC 能够生成逼真的图像，并且与传统的 EBM 或 MCMC 不同，学习的短程 MCMC 能够重新构建观察图像并在图像之间插值，就像生成器或流模型一样。