Score matching is a popular method for estimating unnormalized statistical
models. However, it has been so far limited to simple, shallow models or
low-dimensional data, due to the difficulty of computing the Hessian of
log-density functions. We show this difficulty can be mitigated by projecting
the scores onto random vectors before comparing them. This objective, called
sliced score matching, only involves Hessian-vector products, which can be
easily implemented using reverse-mode automatic differentiation. Therefore,
sliced score matching is amenable to more complex models and higher dimensional
data compared to score matching. Theoretically, we prove the consistency and
asymptotic normality of sliced score matching estimators. Moreover, we
demonstrate that sliced score matching can be used to learn deep score
estimators for implicit distributions. In our experiments, we show sliced score
matching can learn deep energy-based models effectively, and can produce
accurate score estimates for applications such as variational inference with
implicit distributions and training Wasserstein Auto-Encoders.

本研究提出了一种新的分数匹配方法，即 sliced score matching，该方法仅涉及 Hessian-vector product，在更复杂的模型和高维数据上具有良好的鲁棒性和可扩展性，并可用于学习深层次的隐式分布模型和训练 Wasserstein Auto-Encoders 模型。