In deep learning (DL) the instability phenomenon is widespread and well
documented, most commonly using the classical measure of stability, the
Lipschitz constant. While a small Lipchitz constant is traditionally viewed as
guarantying stability, it does not capture the instability phenomenon in DL for
classification well. The reason is that a classification function -- which is
the target function to be approximated -- is necessarily discontinuous, thus
having an 'infinite' Lipchitz constant. As a result, the classical approach
will deem every classification function unstable, yet basic classification
functions a la 'is there a cat in the image?' will typically be locally very
'flat' -- and thus locally stable -- except at the decision boundary. The lack
of an appropriate measure of stability hinders a rigorous theory for stability
in DL, and consequently, there are no proper approximation theoretic results
that can guarantee the existence of stable networks for classification
functions. In this paper we introduce a novel stability measure
$\mathscr{S}(f)$, for any classification function $f$, appropriate to study the
stability of discontinuous functions and their approximations. We further prove
two approximation theorems: First, for any $\epsilon > 0$ and any
classification function $f$ on a \emph{compact set}, there is a neural network
(NN) $\psi$, such that $\psi - f \neq 0$ only on a set of measure $< \epsilon$,
moreover, $\mathscr{S}(\psi) \geq \mathscr{S}(f) - \epsilon$ (as accurate and
stable as $f$ up to $\epsilon$). Second, for any classification function $f$
and $\epsilon > 0$, there exists a NN $\psi$ such that $\psi = f$ on the set of
points that are at least $\epsilon$ away from the decision boundary.

在深度学习中，我们引入了一种新的稳定性测量方法，适用于研究不连续函数及其逼近的稳定性。通过证明两个逼近定理，我们展示了对于任何分类函数，都可以构建一个神经网络使其在小于一定误差的测度内准确且稳定，同时也可以构建一个神经网络使其与分类函数在离决策边界至少一定距离的点上相等。

分类问题中是否存在稳定的神经网络？-- 人工智能稳定性的新视角

Do stable neural networks exist for classification problems? -- A new  view on stability in AI

There is an emerging interest in generating robust counterfactual
explanations that would remain valid if the model is updated or changed even
slightly. Towards finding robust counterfactuals, existing literature often
assumes that the original model $m$ and the new model $M$ are bounded in the
parameter space, i.e., $\|\text{Params}(M){-}\text{Params}(m)\|{<}\Delta$.
However, models can often change significantly in the parameter space with
little to no change in their predictions or accuracy on the given dataset. In
this work, we introduce a mathematical abstraction termed
\emph{naturally-occurring} model change, which allows for arbitrary changes in
the parameter space such that the change in predictions on points that lie on
the data manifold is limited. Next, we propose a measure -- that we call
\emph{Stability} -- to quantify the robustness of counterfactuals to potential
model changes for differentiable models, e.g., neural networks. Our main
contribution is to show that counterfactuals with sufficiently high value of
\emph{Stability} as defined by our measure will remain valid after potential
``naturally-occurring'' model changes with high probability (leveraging
concentration bounds for Lipschitz function of independent Gaussians). Since
our quantification depends on the local Lipschitz constant around a data point
which is not always available, we also examine practical relaxations of our
proposed measure and demonstrate experimentally how they can be incorporated to
find robust counterfactuals for neural networks that are close, realistic, and
remain valid after potential model changes.

通过引入 “自然发生的模型变化” 这种数学抽象概念，提出了一种能度量不同模型下反事实解释的鲁棒性的度量标准，该标准称为 “稳定性”，并证明了当该度量标准足够高时，针对高斯函数的稳定性也足够高，能够有效解决不同模型之间的反事实解释一致性问题。