Structured distributions, i.e. distributions over combinatorial spaces, are
commonly used to learn latent probabilistic representations from observed data.
However, scaling these models is bottlenecked by the high computational and
memory complexity with respect to the size of the latent representations.
Common models such as Hidden Markov Models (HMMs) and Probabilistic
Context-Free Grammars (PCFGs) require time and space quadratic and cubic in the
number of hidden states respectively. This work demonstrates a simple approach
to reduce the computational and memory complexity of a large class of
structured models. We show that by viewing the central inference step as a
matrix-vector product and using a low-rank constraint, we can trade off model
expressivity and speed via the rank. Experiments with neural parameterized
structured models for language modeling, polyphonic music modeling,
unsupervised grammar induction, and video modeling show that our approach
matches the accuracy of standard models at large state spaces while providing
practical speedups.

本文提出一种简单的方法，通过将中心推理步骤视为矩阵向量积并使用低秩约束，以降低大类结构模型的计算和内存复杂度。使用神经参数化结构模型进行语言建模，多声部音乐建模，无监督语法归纳和视频建模的实验表明，我们的方法与大型状态空间的标准模型具有相同的准确性，同时提供了实际的加速。