Deep dynamic generative models are developed to learn sequential dependencies
in time-series data. The multi-layered model is designed by constructing a
hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential
stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden
state, inherited from the previous SBNs in the sequence, and is used to
regulate its hidden bias. Scalable learning and inference algorithms are
derived by introducing a recognition model that yields fast sampling from the
variational posterior. This recognition model is trained jointly with the
generative model, by maximizing its variational lower bound on the
log-likelihood. Experimental results on bouncing balls, polyphonic music,
motion capture, and text streams show that the proposed approach achieves
state-of-the-art predictive performance, and has the capacity to synthesize
various sequences.

通过构建时序 Sigmoid 置信网络的层次结构，设计深动态生成模型，引入识别模型从变分后验中快速采样，提出的方法在多种序列数据上取得最先进的预测性能，能够合成各种序列。