Even though probabilistic treatments of neural networks have a long history,
they have not found widespread use in practice. Sampling approaches are often
too slow already for simple networks. The size of the inputs and the depth of
typical CNN architectures in computer vision only compound this problem.
Uncertainty in neural networks has thus been largely ignored in practice,
despite the fact that it may provide important information about the
reliability of predictions and the inner workings of the network. In this
paper, we introduce two lightweight approaches to making supervised learning
with probabilistic deep networks practical: First, we suggest probabilistic
output layers for classification and regression that require only minimal
changes to existing networks. Second, we employ assumed density filtering and
show that activation uncertainties can be propagated in a practical fashion
through the entire network, again with minor changes. Both probabilistic
networks retain the predictive power of the deterministic counterpart, but
yield uncertainties that correlate well with the empirical error induced by
their predictions. Moreover, the robustness to adversarial examples is
significantly increased.

本文介绍了两种轻量级的方法，使具有概率性的深层网络的监督学习变得实用，首先，我们建议使用分类和回归的概率性输出层，其对现有网络只需进行最小的更改，其次，我们使用假设密度过滤（assumed density filtering）并展示如何将激活不确定性以实用的方式传播到整个网络中，这两种概率性网络保留了确定性网络的预测能力，但产生与其预测引起的经验误差相关的不确定性。