Energy-Based Models (EBMs) are an important class of probabilistic models,
also known as random fields and undirected graphical models. EBMs are
un-normalized and thus radically different from other popular self-normalized
probabilistic models such as hidden Markov models (HMMs), autoregressive
models, generative adversarial nets (GANs) and variational auto-encoders
(VAEs). Over the past years, EBMs have attracted increasing interest not only
from the core machine learning community, but also from application domains
such as speech, vision, natural language processing (NLP) and so on, due to
significant theoretical and algorithmic progress. The sequential nature of
speech and language also presents special challenges and needs a different
treatment from processing fix-dimensional data (e.g., images). Therefore, the
purpose of this monograph is to present a systematic introduction to
energy-based models, including both algorithmic progress and applications in
speech and language processing. First, the basics of EBMs are introduced,
including classic models, recent models parameterized by neural networks,
sampling methods, and various learning methods from the classic learning
algorithms to the most advanced ones. Then, the application of EBMs in three
different scenarios is presented, i.e., for modeling marginal, conditional and
joint distributions, respectively. 1) EBMs for sequential data with
applications in language modeling, where the main focus is on the marginal
distribution of a sequence itself; 2) EBMs for modeling conditional
distributions of target sequences given observation sequences, with
applications in speech recognition, sequence labeling and text generation; 3)
EBMs for modeling joint distributions of both sequences of observations and
targets, and their applications in semi-supervised learning and calibrated
natural language understanding.

能量模型（EBM）是一种重要的概率模型，也被称为随机场和无向图模型。EBM 是非标准化的，与其他流行的自标准化概率模型（如隐马尔可夫模型（HMMs）、自回归模型、生成对抗网络（GANs）和变分自动编码器（VAEs））截然不同。近年来，由于在理论和算法方面取得了重大进展，EBM 不仅受到核心机器学习领域的越来越多的关注，而且还受到了语音、视觉、自然语言处理（NLP）等应用领域的关注。语音和语言的序列性质也带来了特殊的挑战，并需要与处理固定维度数据（如图像）有所不同的处理方法。因此，本文的目的是系统介绍能量基模型，包括算法进展和在语音和语言处理中的应用。首先，介绍了 EBM 的基础知识，包括经典模型、最近由神经网络参数化的模型、采样方法以及从经典学习算法到最先进算法的各种学习方法。然后，介绍了 EBM 在三种不同场景下的应用，即建模边际、条件和联合分布。1）用于语言建模的序列数据的 EBM 应用，其中主要关注的是序列本身的边际分布；2）用于建模给定观测序列条件分布的 EBM，在语音识别、序列标注和文本生成方面有应用；3）用于建模观测序列和目标序列的联合分布的 EBM 及其在半监督学习和校准自然语言理解方面的应用。