In many applications with real-world consequences, it is crucial to develop reliable uncertainty estimation for the predictions made by the AI decision systems. Targeting at the goal of estimating uncertainty, various deep neural network (DNN) based uncertainty estimation algorithms have been proposed. However, the robustness of the uncertainty returned by these algorithms has not been systematically explored. In this work, to raise the awareness of the research community on robust uncertainty estimation, we show that state-of-the-art uncertainty estimation algorithms could fail catastrophically under our proposed adversarial attack despite their impressive performance on uncertainty estimation. In particular, we aim at attacking the out-domain uncertainty estimation: under our attack, the uncertainty model would be fooled to make high-confident predictions for the out-domain data, which they originally would have rejected. Extensive experimental results on various benchmark image datasets show that the uncertainty estimated by state-of-the-art methods could be easily corrupted by our attack.

本文研究了基于深度神经网络的不确定性估计算法的鲁棒性，并提出了一种敌对攻击方法，证明了这些算法对于出域数据的不确定性估计容易受到攻击。在各种基准图像数据集上进行的大量实验结果表明，最先进方法所估计的不确定性很容易被我们的攻击所破坏。

深度神经网络中攻击领域外不确定性估计