We apply concepts from manifold regularization to develop new regularization
techniques for training locally stable deep neural networks. Our regularizers
are based on a sparsification of the graph Laplacian which holds with high
probability when the data is sparse in high dimensions, as is common in deep
learning. Empirically, our networks exhibit stability in a diverse set of
perturbation models, including $\ell_2$, $\ell_\infty$, and Wasserstein-based
perturbations; in particular, we achieve 40% adversarial accuracy on CIFAR-10
against an adaptive PGD attack using $\ell_\infty$ perturbations of size
$\epsilon = 8/255$, and state-of-the-art verified accuracy of 21% in the same
perturbation model. Furthermore, our techniques are efficient, incurring
overhead on par with two additional parallel forward passes through the
network.

本文使用流形正则化的概念开发了新的正则化技术，以训练具有局部稳定性的深度神经网络；我们的正则化器基于图拉普拉斯矩阵的稀疏化，当数据在高维空间中稀疏时，能够高概率地保持；经验证明，我们的网络表现出稳定性，并能够在不同的扰动模型下经受良好的检验；此外，我们的技术高效，并且与网络的额外两个并行前向传递的开销相当。