Deep generative networks provide a powerful tool for modeling complex data in
a wide range of applications. In inverse problems that use these networks as
generative priors on data, one must often perform inference of the inputs of
the networks from the outputs. Inference is also required for sampling during
stochastic training on these generative models. This paper considers inference
in a deep stochastic neural network where the parameters (e.g., weights, biases
and activation functions) are known and the problem is to estimate the values
of the input and hidden units from the output. While several approximate
algorithms have been proposed for this task, there are few analytic tools that
can provide rigorous guarantees in the reconstruction error. This work presents
a novel and computationally tractable output-to-input inference method called
Multi-Layer Vector Approximate Message Passing (ML-VAMP). The proposed
algorithm, derived from expectation propagation, extends earlier AMP methods
that are known to achieve the replica predictions for optimality in simple
linear inverse problems. Our main contribution shows that the mean-squared
error (MSE) of ML-VAMP can be exactly predicted in a certain large system limit
(LSL) where the numbers of layers is fixed and weight matrices are random and
orthogonally-invariant with dimensions that grow to infinity. ML-VAMP is thus a
principled method for output-to-input inference in deep networks with a
rigorous and precise performance achievability result in high dimensions.

本文介绍了一种深度随机神经网络的推理方法，称为多层向量近似信息传递（ML-VAMP），通过推导模型的均方误差（MSE），它在高维度下具有严格的性能可达性结果。