Conformal Prediction (CP) is a popular uncertainty quantification method that
provides distribution-free, statistically valid prediction sets, assuming that
training and test data are exchangeable. In such a case, CP's prediction sets
are guaranteed to cover the (unknown) true test output with a user-specified
probability. Nevertheless, this guarantee is violated when the data is
subjected to adversarial attacks, which often result in a significant loss of
coverage. Recently, several approaches have been put forward to recover CP
guarantees in this setting. These approaches leverage variations of randomised
smoothing to produce conservative sets which account for the effect of the
adversarial perturbations. They are, however, limited in that they only support
$\ell^2$-bounded perturbations and classification tasks. This paper introduces
\emph{VRCP (Verifiably Robust Conformal Prediction)}, a new framework that
leverages recent neural network verification methods to recover coverage
guarantees under adversarial attacks. Our VRCP method is the first to support
perturbations bounded by arbitrary norms including $\ell^1$, $\ell^2$, and
$\ell^\infty$, as well as regression tasks. We evaluate and compare our
approach on image classification tasks (CIFAR10, CIFAR100, and TinyImageNet)
and regression tasks for deep reinforcement learning environments. In every
case, VRCP achieves above nominal coverage and yields significantly more
efficient and informative prediction regions than the SotA.

使用最新的神经网络验证方法，基于 VRCP（可验证鲁棒性适应预测）框架，本文提出了一种新的方法，用于恢复在遭受对抗攻击时的预测覆盖率保证，并支持任意范数的扰动和回归任务，实验结果表明，在图像分类和强化学习环境的回归任务中，VRCP 方法达到了超过标称覆盖率的结果，并且比目前最先进的方法更高效且信息量更丰富。

可验证的强健拟合预测

Verifiably Robust Conformal Prediction

Randomness supports many critical functions in the field of machine learning
(ML) including optimisation, data selection, privacy, and security. ML systems
outsource the task of generating or harvesting randomness to the compiler, the
cloud service provider or elsewhere in the toolchain. Yet there is a long
history of attackers exploiting poor randomness, or even creating it -- as when
the NSA put backdoors in random number generators to break cryptography. In
this paper we consider whether attackers can compromise an ML system using only
the randomness on which they commonly rely. We focus our effort on Randomised
Smoothing, a popular approach to train certifiably robust models, and to
certify specific input datapoints of an arbitrary model. We choose Randomised
Smoothing since it is used for both security and safety -- to counteract
adversarial examples and quantify uncertainty respectively. Under the hood, it
relies on sampling Gaussian noise to explore the volume around a data point to
certify that a model is not vulnerable to adversarial examples. We demonstrate
an entirely novel attack against it, where an attacker backdoors the supplied
randomness to falsely certify either an overestimate or an underestimate of
robustness. We demonstrate that such attacks are possible, that they require
very small changes to randomness to succeed, and that they can be hard to
detect. As an example, we hide an attack in the random number generator and
show that the randomness tests suggested by NIST fail to detect it. We advocate
updating the NIST guidelines on random number testing to make them more
appropriate for safety-critical and security-critical machine-learning
applications.

本文考虑攻击者是否可以只利用制造机器学习模型所依赖的随机性来破坏模型的安全性， 发现攻击者能够利用 Randomised Smoothing，一种用于提高模型抵抗对抗性攻击和量化不确定性的方法，背后基于对高斯噪声采样，来进行欺骗性认证，而且攻击只需要更改极小的随机数。因此，作者提出更新 NIST 的随机数测试准则，以使其更适用于安全和关键性的机器学习应用。