Ensemble approaches for uncertainty estimation have recently been applied to the tasks of misclassification detection, out-of-distribution input detection and adversarial attack detection. Prior Networks have been proposed as an approach to efficiently emulating an ensemble of models by parameterising a Dirichlet prior distribution over output distributions. These models have been shown to outperform ensemble approaches, such as Monte-Carlo Dropout, on the task of out-of-distribution input detection. However, scaling Prior Networks to complex datasets with many classes is difficult using the training criteria originally proposed. This paper makes two contributions. Firstly, we show that the appropriate training criterion for Prior Networks is the reverse KL-divergence between Dirichlet distributions. Using this loss we successfully train Prior Networks on image classification datasets with up to 200 classes and improve out-of-distribution detection performance. Secondly, taking advantage of the new training criterion, this paper investigates using Prior Networks to detect adversarial attacks. It is shown that the construction of successful adaptive whitebox attacks, which affect the prediction and evade detection, against Prior Networks trained on CIFAR-10 and CIFAR-100 takes a greater amount of computational effort than against standard neural networks, adversarially trained neural networks and dropout-defended networks.

本文研究了如何使用先验网络来代替集成模型在分类、异常检测和对抗攻击检测任务上，并使用反向KL散度作为训练标准，提出了一种新的对抗训练方法。研究表明，在分类数据集上，使用先验网络相比于蒙特卡洛dropout等其他集成方法能够更好地检测出数据集之外的输入，但是对抗攻击会对其造成影响，需要进行更为复杂的对抗训练来提高其鲁棒性。

先验网络的反向KL散度训练：提高不确定性和对抗鲁棒性