Among Bayesian methods, Monte-Carlo dropout provides principled tools for
evaluating the epistemic uncertainty of neural networks. Its popularity
recently led to seminal works that proposed activating the dropout layers only
during inference for evaluating uncertainty. This approach, which we call
dropout injection, provides clear benefits over its traditional counterpart
(which we call embedded dropout) since it allows one to obtain a post hoc
uncertainty measure for any existing network previously trained without
dropout, avoiding an additional, time-consuming training process.
Unfortunately, no previous work compared injected and embedded dropout;
therefore, we provide the first thorough investigation, focusing on regression
problems. The main contribution of our work is to provide guidelines on the
effective use of injected dropout so that it can be a practical alternative to
the current use of embedded dropout. In particular, we show that its
effectiveness strongly relies on a suitable scaling of the corresponding
uncertainty measure, and we discuss the trade-off between negative
log-likelihood and calibration error as a function of the scale factor.
Experimental results on UCI data sets and crowd counting benchmarks support our
claim that dropout injection can effectively behave as a competitive post hoc
uncertainty quantification technique.

本文研究了贝叶斯方法中的一种蒙特卡罗 dropout，提供了在评估神经网络的认知不确定性方面的原则性工具，继而提出了 dropout 注入的方法，它是嵌入式 dropout 的实用替代方案，文中着重研究了回归问题的实验结果，同时指出了其有效使用的指南。