The application of neural network models to scientific machine learning tasks
has proliferated in recent years. In particular, neural network models have
proved to be adept at modeling processes with spatial-temporal complexity.
Nevertheless, these highly parameterized models have garnered skepticism in
their ability to produce outputs with quantified error bounds over the regimes
of interest. Hence there is a need to find uncertainty quantification methods
that are suitable for neural networks. In this work we present comparisons of
the parametric uncertainty quantification of neural networks modeling complex
spatial-temporal processes with Hamiltonian Monte Carlo and Stein variational
gradient descent and its projected variant. Specifically we apply these methods
to graph convolutional neural network models of evolving systems modeled with
recurrent neural network and neural ordinary differential equations
architectures. We show that Stein variational inference is a viable alternative
to Monte Carlo methods with some clear advantages for complex neural network
models. For our exemplars, Stein variational interference gave similar
uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit
with generally more generous variance.Projected Stein variational gradient
descent also produced similar uncertainty profiles to the non-projected
counterpart, but large reductions in the active weight space were confounded by
the stability of the neural network predictions and the convoluted likelihood
landscape.

神经网络模型在科学机器学习任务中的应用近年来已大量增加。尤其是，在建模具有时空复杂性的过程方面，神经网络模型表现出色。然而，这些高度参数化的模型在产生与感兴趣的区域保持定量化误差界限方面引起了怀疑。因此，有必要寻找适用于神经网络的不确定性量化方法。本研究对神经网络对复杂时空过程进行了参数不确定性量化的比较，采用了哈密顿蒙特卡洛和 Stein 变分梯度下降及其投影变体。具体而言，我们将这些方法应用于用递归神经网络和神经常微分方程模型建模的发展系统的图卷积神经网络模型。我们展示了 Stein 变分推断是复杂神经网络模型的可行替代方法。对于我们的示例，与哈密顿蒙特卡洛相比，Stein 变分推断在时间上给出了类似的不确定性描述，尽管方差普遍更大。投影 Stein 变分梯度下降也产生了与非投影对应的类似的不确定性描述，但是在神经网络预测的稳定性和复杂的似然函数空间中卷积产生了活动权重空间的大幅减少与困扰。

图卷积神经网络模型下演化过程的不确定性量化

Uncertainty Quantification of Graph Convolution Neural Network Models of  Evolving Processes

This paper studies the dynamic generator model for spatial-temporal processes
such as dynamic textures and action sequences in video data. In this model,
each time frame of the video sequence is generated by a generator model, which
is a non-linear transformation of a latent state vector, where the non-linear
transformation is parametrized by a top-down neural network. The sequence of
latent state vectors follows a non-linear auto-regressive model, where the
state vector of the next frame is a non-linear transformation of the state
vector of the current frame as well as an independent noise vector that
provides randomness in the transition. The non-linear transformation of this
transition model can be parametrized by a feedforward neural network. We show
that this model can be learned by an alternating back-propagation through time
algorithm that iteratively samples the noise vectors and updates the parameters
in the transition model and the generator model. We show that our training
method can learn realistic models for dynamic textures and action patterns.

本研究研究了动态生成模型的空间 - 时间过程，如视频数据中的动态纹理和动作序列，模型通过一个生成模型产生视频序列中的每个时间帧，生成模型是潜在状态向量的非线性转换，并由自回归模型控制。