Uncertainty estimation is an essential step in the evaluation of the
robustness for deep learning models in computer vision, especially when applied
in risk-sensitive areas. However, most state-of-the-art deep learning models
either fail to obtain uncertainty estimation or need significant modification
(e.g., formulating a proper Bayesian treatment) to obtain it. Most previous
methods are not able to take an arbitrary model off the shelf and generate
uncertainty estimation without retraining or redesigning it. To address this
gap, we perform a systematic exploration into training-free uncertainty
estimation for dense regression, an unrecognized yet important problem, and
provide a theoretical construction justifying such estimations. We propose
three simple and scalable methods to analyze the variance of outputs from a
trained network under tolerable perturbations: infer-transformation,
infer-noise, and infer-dropout. They operate solely during the inference,
without the need to re-train, re-design, or fine-tune the models, as typically
required by state-of-the-art uncertainty estimation methods. Surprisingly, even
without involving such perturbations in training, our methods produce
comparable or even better uncertainty estimation when compared to
training-required state-of-the-art methods.

提出三种方法以分析密集回归训练网络的输出方差，在推理过程中无需重新训练或调整模型，这些方法得出的不确定性估计结果与需要重新训练以及调整的最先进的方法相当甚至更好。

基于敏感度的密集回归无需训练的不确定性估计

Training-Free Uncertainty Estimation for Dense Regression: Sensitivity  as a Surrogate

Principal component analysis (PCA) is one of the most powerful tools in
machine learning. The simplest method for PCA, the power iteration, requires
$\mathcal O(1/\Delta)$ full-data passes to recover the principal component of a
matrix with eigen-gap $\Delta$. Lanczos, a significantly more complex method,
achieves an accelerated rate of $\mathcal O(1/\sqrt{\Delta})$ passes. Modern
applications, however, motivate methods that only ingest a subset of available
data, known as the stochastic setting. In the online stochastic setting, simple
algorithms like Oja's iteration achieve the optimal sample complexity $\mathcal
O(\sigma^2/\Delta^2)$. Unfortunately, they are fully sequential, and also
require $\mathcal O(\sigma^2/\Delta^2)$ iterations, far from the $\mathcal
O(1/\sqrt{\Delta})$ rate of Lanczos. We propose a simple variant of the power
iteration with an added momentum term, that achieves both the optimal sample
and iteration complexity. In the full-pass setting, standard analysis shows
that momentum achieves the accelerated rate, $\mathcal O(1/\sqrt{\Delta})$. We
demonstrate empirically that naively applying momentum to a stochastic method,
does not result in acceleration. We perform a novel, tight variance analysis
that reveals the "breaking-point variance" beyond which this acceleration does
not occur. By combining this insight with modern variance reduction techniques,
we construct stochastic PCA algorithms, for the online and offline setting,
that achieve an accelerated iteration complexity $\mathcal O(1/\sqrt{\Delta})$.
Due to the embarassingly parallel nature of our methods, this acceleration
translates directly to wall-clock time if deployed in a parallel environment.
Our approach is very general, and applies to many non-convex optimization
problems that can now be accelerated using the same technique.

我们提出了一种简单的变种 Power Iteration 方法，使用动量项，既实现了最优的 PCA 速率和迭代复杂度，也适用于随机数据集，并利用现代方差缩减技术，加速了很多非凸优化问题。