Certifying the robustness of model performance under bounded data
distribution drifts has recently attracted intensive interest under the
umbrella of distributional robustness. However, existing techniques either make
strong assumptions on the model class and loss functions that can be certified,
such as smoothness expressed via Lipschitz continuity of gradients, or require
to solve complex optimization problems. As a result, the wider application of
these techniques is currently limited by its scalability and flexibility --
these techniques often do not scale to large-scale datasets with modern deep
neural networks or cannot handle loss functions which may be non-smooth such as
the 0-1 loss. In this paper, we focus on the problem of certifying
distributional robustness for blackbox models and bounded loss functions, and
propose a novel certification framework based on the Hellinger distance. Our
certification technique scales to ImageNet-scale datasets, complex models, and
a diverse set of loss functions. We then focus on one specific application
enabled by such scalability and flexibility, i.e., certifying out-of-domain
generalization for large neural networks and loss functions such as accuracy
and AUC. We experimentally validate our certification method on a number of
datasets, ranging from ImageNet, where we provide the first non-vacuous
certified out-of-domain generalization, to smaller classification tasks where
we are able to compare with the state-of-the-art and show that our method
performs considerably better.

本研究提出了一种基于 Hellinger 距离的证明技术，可以用于证明黑盒模型和有界损失函数的分布健壮性和域外泛化。实验证明这种技术在大型神经网络和不同类型的损失函数下具有良好的可扩展性和灵活性。

黑盒函数的域外泛化认证

Certifying Out-of-Domain Generalization for Blackbox Functions

The ability to interpret machine learning models has become increasingly
important now that machine learning is used to inform consequential decisions.
We propose an approach called model extraction for interpreting complex,
blackbox models. Our approach approximates the complex model using a much more
interpretable model; as long as the approximation quality is good, then
statistical properties of the complex model are reflected in the interpretable
model. We show how model extraction can be used to understand and debug random
forests and neural nets trained on several datasets from the UCI Machine
Learning Repository, as well as control policies learned for several classical
reinforcement learning problems.

这篇论文提出一种名为模型抽取的方法，通过构建一个可解释程度更高的模型来近似黑箱模型，从而理解和调试机器学习模型在各种数据集上训练的结果，并在经典强化学习问题中学习控制策略。